-- Logs begin at Tue 2022-09-13 07:18:57 UTC, end at Tue 2022-09-13 08:23:37 UTC. -- Sep 13 07:44:27.464720 dv-cinder-857075-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 07:44:27.472521 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 07:44:27 2022] *** Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: nodename: dv-cinder-857075-3-iscsi Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: machine: x86_64 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: clock source: unix Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: pcre jit disabled Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: detected number of CPU cores: 4 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: current working directory: / Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: detected binary path: /usr/bin/uwsgi-core Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: your processes number limit is 47680 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: your memory page size is 4096 bytes Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: detected max file descriptor number: 2048 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: lock engine: pthread robust mutexes Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: thunder lock: enabled Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 07:44:27.477758 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 07:44:27.515039 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: Python main interpreter initialized at 0x559695a629c0 Sep 13 07:44:27.515039 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: python threads support enabled Sep 13 07:44:27.515039 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: your server socket listen backlog is limited to 100 connections Sep 13 07:44:27.515039 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: your mercy for graceful operations on workers is 90 seconds Sep 13 07:44:27.515402 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 07:44:27.515479 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: *** Operational MODE: preforking *** Sep 13 07:44:27.515736 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: *** uWSGI is running in multiple interpreter mode *** Sep 13 07:44:27.515736 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: spawned uWSGI master process (pid: 81439) Sep 13 07:44:27.515937 dv-cinder-857075-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 07:44:27.517017 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: spawned uWSGI worker 1 (pid: 81451, cores: 1) Sep 13 07:44:27.517017 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: spawned uWSGI worker 2 (pid: 81452, cores: 1) Sep 13 07:44:27.517017 dv-cinder-857075-3-iscsi devstack@keystone.service[81439]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 07:44:31.789283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 07:44:31.789283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: from cryptography.utils import int_from_bytes Sep 13 07:44:31.790106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 07:44:31.790106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: from cryptography.utils import int_from_bytes Sep 13 07:44:31.790106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 07:44:31.790106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: from cryptography.utils import int_from_bytes Sep 13 07:44:31.790606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 07:44:31.790606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: from cryptography.utils import int_from_bytes Sep 13 07:44:31.839961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 07:44:31.840153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 07:44:31.840153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 07:44:31.840327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 07:44:31.840407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] command line args: [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 07:44:31.840479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] command line args: [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 07:44:31.840589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 07:44:31.840678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 07:44:31.840757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ================================================================================ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 07:44:31.840833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ================================================================================ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 07:44:31.841028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] admin_token = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] admin_token = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_dir = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_dir = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] config_source = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] config_source = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] debug = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.841914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] debug = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.842902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_config_append = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_config_append = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_dir = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.843980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_dir = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.844956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.845940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.846978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.847136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.847255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.847413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.847499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.847904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] publish_errors = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] publish_errors = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.848934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] standard_threads = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] standard_threads = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.849987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_journal = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_journal = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_json = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_json = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_stderr = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_stderr = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] use_syslog = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] use_syslog = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.850975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.851055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 07:44:31.851230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.851965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.852936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.853946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.854922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.855978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.856935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.857982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.858946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.859917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.860859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.861951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.862977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.863961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.864904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.865989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.866936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.867995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.868963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.869935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.870897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.871845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.872021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.872099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.872769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.872854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.873940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.874991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.875927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.876906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.external = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.external = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.877885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.password = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.password = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] auth.token = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.878876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] auth.token = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.879996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.880959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.881696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.882983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.883921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.884937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.885993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.886991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.887984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.888605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.888730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.888819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.888968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.889975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.890997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.891988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892658 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.892972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893358 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.893917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.894873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.895848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.896963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.897878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.898889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.899873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.900985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.901989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.902924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.903991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.904973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.905942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.906924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.907892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.908992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.909992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.910962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.911986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.912961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.913984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.914959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.915699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.916851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.917867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.918976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.919994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.920932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.921948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.922922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.923966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.924937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.925952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.926898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.927858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.928968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 07:44:31.929590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 07:44:31.929661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 07:44:31.959812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.960040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.960207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.960294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.960445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.960677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.968587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.968966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.996441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:31.996670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.000116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.000530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.050989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.051420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.099763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.099934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.100026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.100181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.254957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.255926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.256061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.256145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.256279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.279987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.280071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.280205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.280288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.280422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.280514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.291532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.291633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.304957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81451) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.305877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 07:44:32.323002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.323215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 07:44:32.323563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.323649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.323767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 07:44:32.323873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 07:44:32.324045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 07:44:32.324569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.324911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.325136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.325428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.334451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.334733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.335855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.342397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.342979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.345317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.345605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.345914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.346001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.346840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.347935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.348131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.348214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.348402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.348674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.370978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.371579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.373089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.373376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.373621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.373943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.378041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.378327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.378582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.378664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.378868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.379154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.384881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.385172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.385406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.385695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.390966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.391265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.391483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.391565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.391784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.391877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.392983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.393181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.393457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.408739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.409019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.409369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.409652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.412863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.413511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.416112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.416415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.416743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.417045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.421255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.421811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.424366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.424880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.427527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.427830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428358 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.428929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.429009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.429211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.429494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.444598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.444890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.445165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.445245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.445433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.445527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.445718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.446014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.452193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.452746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.453707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.454286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.455224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.455502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.455774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.456053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.458141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.458437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.458714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.458795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.459981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.471886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.472175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.472463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.472543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.472752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.472983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.473140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.473442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.473529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.473780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.473887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.474215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.474518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.474810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.489505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.490839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.492427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.493754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.495345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.496625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.496844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.498247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.498340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.498591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.498995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.499701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.499996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.500399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.505321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.506727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.506820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.508187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.509650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.510077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.510367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.510652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.511164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.511568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.511855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.512135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.518499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.518894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.519093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.520137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.520591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.520757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.525957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.526211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.526484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.526768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.527047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.527335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.527612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.527899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.528232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.528425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81451) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 07:44:32.529056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.529262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.529435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.529602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.529777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.529961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.530978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.531294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.531544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.531866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.532178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.532528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81451) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 07:44:32.565118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559695a629c0 pid: 81452 (default app) Sep 13 07:44:32.568478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75035-8a4c-44cd-acee-beeb11ef0037 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:32.568767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75035-8a4c-44cd-acee-beeb11ef0037 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:32.569041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e75035-8a4c-44cd-acee-beeb11ef0037 None None] PATH_INFO: `/v3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:32.569346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559695a629c0 pid: 81451 (default app) Sep 13 07:44:32.573184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Tue Sep 13 07:44:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:39.334118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:39.334643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:39.334643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:39.338904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:44:39.344716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:39.345237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:39.345237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:39.418005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: INFO dbcounter [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] Registered counter for database keystone Sep 13 07:44:39.437977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG oslo_db.sqlalchemy.engines [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81452) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 07:44:39.444166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writer thread running {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 07:44:39.620594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG passlib.handlers.bcrypt [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81452) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 07:44:39.620903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG passlib.handlers.bcrypt [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81452) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 07:44:39.678383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:39.679989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:39.809078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 07:44:40.270054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:40.339469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: INFO dbcounter [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] Registered counter for database keystone Sep 13 07:44:40.361401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG oslo_db.sqlalchemy.engines [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 07:44:40.367527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writer thread running {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 07:44:40.484836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e772-136e-4cd2-bc60-6f7a60504a37 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:40.489016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:40.489631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:40.489931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:40.490200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:40.499510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:40.552699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 07:44:40.553044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7af5d-7595-48c9-95cf-1fa27acbfd26 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:40.563694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Sep 13 07:44:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:44:40.569570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:40.586229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361fcb1-86bc-4e57-ba4a-bbe1608e4f41 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:40.590667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:40.591771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:40.592053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:40.592323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] PATH_INFO: `/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:40.593942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] RBAC: Authorizing `identity:get_project(project_id=dab10ae977dc434f9d22b1fb83ca19e8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:40.646376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 07:44:40.646726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850f7fb-4ccd-43a2-be36-1dbae359b3ca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:40.651266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Sep 13 07:44:40 2022] GET /identity/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:42.764713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed850-bcb0-4362-83ef-79408941bbca None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:42.765319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed850-bcb0-4362-83ef-79408941bbca None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:42.765319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed850-bcb0-4362-83ef-79408941bbca None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:42.769325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:42.774828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339145-c16b-4961-8c88-944bb753f831 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:42.775211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339145-c16b-4961-8c88-944bb753f831 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:42.775374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339145-c16b-4961-8c88-944bb753f831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:42.795641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-80339145-c16b-4961-8c88-944bb753f831 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:42.797140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-80339145-c16b-4961-8c88-944bb753f831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:42.804854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 07:44:43.068455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ed850-bcb0-4362-83ef-79408941bbca None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:43.080399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ed850-bcb0-4362-83ef-79408941bbca None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:43.084545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:43.085145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:43.085429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:43.085699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:43.093681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:43.095464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d88216-ff8e-444b-9d41-11cfaed0f932 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:43.104010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Sep 13 07:44:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:44:43.109687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339145-c16b-4961-8c88-944bb753f831 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:43.118722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339145-c16b-4961-8c88-944bb753f831 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:43.123031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:43.123657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:43.123940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:43.124215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:43.125524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:43.127400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f953240-c0da-48a3-a91b-bbedc1660aee admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:43.128913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Tue Sep 13 07:44:43 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:45.350624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486392ea-ac96-4649-9336-de1e3fd420d7 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:45.351164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486392ea-ac96-4649-9336-de1e3fd420d7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:45.351264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486392ea-ac96-4649-9336-de1e3fd420d7 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:45.352589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:45.357799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:45.358342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:45.358915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:45.377435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:45.378651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:45.412868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c1ab-2104-45e5-8ebb-b349854da5d4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:45.413159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c1ab-2104-45e5-8ebb-b349854da5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:45.413441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c1ab-2104-45e5-8ebb-b349854da5d4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:45.414803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:44:45.419973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:45.420264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:45.420542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:45.461840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:45.463816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG passlib.handlers.bcrypt [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81451) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 07:44:45.464216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG passlib.handlers.bcrypt [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81451) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 07:44:45.513454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:45.515054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:45.615198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:44:45.728143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:45.752690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:45.831731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f8c30-924d-43e3-b082-88f6bba736c2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:45.863837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:45.864465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:45.864741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:45.865009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:45.876402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:44:45.885093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:44:45.886018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:44:45.886932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:44:45.893558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:45.893855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:45.975460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:45.994969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2ee13-30b1-43f5-842e-8cfb5d6c4cee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:45.995339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39003dc-c6d9-453b-82d0-e41fc8ff2f48 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:45.998633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:46.029443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.030076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.030351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.030618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.031381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:46.031686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.033065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.034771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.035629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.035910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.036182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.045102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:44:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:46.050748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:46.104321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.106010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.149136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb28d-2a46-4ae7-aeeb-c92c7dde3134 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:46.184541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.185174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.185455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.185759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.187182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.188631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.190226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 9/16] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:44:46 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:46.195538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:46.203042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 8/17] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:44:46.208384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:46.209475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:46.211656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:46.292759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dee7e-199b-4d43-8846-c1e8e31df576 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:46.305367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9479b28-6240-4a02-814c-8681f7295a70 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:46.327874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.328503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.328786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.329057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.330198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.331639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.333158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:46.338841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.338988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:46.339720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.339997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.340827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.358874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.360330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:44:46.378269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:44:46.379333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:44:46.380142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:44:46.380142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:44:46.380142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:44:46.380142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:44:46.385102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:46.422143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:46.439945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:46.440985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8726c-9d47-4db2-bfa3-200e1cd87843 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:46.474417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.475019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.475294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.475564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.476278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.domain.id=default, user.id=d247b00da71b4a5ebefbb06a796143be) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:46.476587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.478042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a630e-ccba-4699-a13e-dda451824bf0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.484971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93b69a-6b87-41b3-a2eb-34aea8478bf9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:46.485174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Sep 13 07:44:46 2022] GET /identity/v3/role_assignments?user.id=d247b00da71b4a5ebefbb06a796143be&scope.domain.id=default&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:46.520719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:46.521353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:46.521631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:46.521940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:46.522604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:46.522907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:46.524342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117fc55-f902-4a11-8672-09ee0698f2a9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:46.544129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:48.877765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717accd-7710-4907-a0d3-6d3498375852 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:48.878292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717accd-7710-4907-a0d3-6d3498375852 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:48.878364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717accd-7710-4907-a0d3-6d3498375852 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:48.880148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:48.889804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73498b2f-6dda-41e5-b342-87c1740d8d44 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:48.890689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73498b2f-6dda-41e5-b342-87c1740d8d44 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:48.891167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73498b2f-6dda-41e5-b342-87c1740d8d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:48.919539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-73498b2f-6dda-41e5-b342-87c1740d8d44 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:48.921076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73498b2f-6dda-41e5-b342-87c1740d8d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:49.014223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:49.020675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679794de-c6a9-46d1-854c-a34c17f597e8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.020957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679794de-c6a9-46d1-854c-a34c17f597e8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.021228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679794de-c6a9-46d1-854c-a34c17f597e8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.022842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:49.028211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95537-efc3-44be-a9fe-2338050416e5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.028743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95537-efc3-44be-a9fe-2338050416e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.029259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e95537-efc3-44be-a9fe-2338050416e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.054847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-73e95537-efc3-44be-a9fe-2338050416e5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:49.056032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73e95537-efc3-44be-a9fe-2338050416e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:49.130526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:49.160400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:49.379890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679794de-c6a9-46d1-854c-a34c17f597e8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:49.509069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dbad9-942b-418f-b9eb-2ad619bb2738 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.509371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dbad9-942b-418f-b9eb-2ad619bb2738 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.509546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dbad9-942b-418f-b9eb-2ad619bb2738 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.511055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 13/26] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:49.521172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.521451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.521722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.523030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679794de-c6a9-46d1-854c-a34c17f597e8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:49.563243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:49.564446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:49.577383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:49.578059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.578339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.578610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.597845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:44:49.604037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:44:49.604872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:44:49.606173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 14/27] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:44:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:44:49.614282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:49.620803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:49.695121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 14/28] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:44:49.722494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:49.781178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc7694-b1ab-4c7e-8578-2c1a91d6ecc9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:49.818971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:49.819585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.819864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.820297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.820970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:49.821269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:49.822711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:49.831878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:44:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:49.837528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:49.901754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c14f1-9426-4cfe-b542-4f909ba4b173 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:49.908187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:49.945768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:49.946413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:49.946882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:49.947167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:49.966085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:49.966602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:49.969227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:44:50.005051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:44:50.006093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:44:50.006902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:44:50.006902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:44:50.006902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:44:50.006902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 15/30] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:44:50.019235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:50.042762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e778ff-3c76-4e39-a503-cc04b42aef59 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.120387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.121081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.121403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.121675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.124258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.125723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.127336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:44:49 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:50.132801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:50.199282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853aa8b6-d7c1-4c49-a10a-2e6393af677b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.238130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.238741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.239015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.239281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.240042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:50.240339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.241719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81241c-5c9b-41b9-8f76-92f47f3fd621 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.253949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 16/32] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:50.255106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788fc1-fb3c-4e03-ba90-352516930b28 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.255420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788fc1-fb3c-4e03-ba90-352516930b28 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.255785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788fc1-fb3c-4e03-ba90-352516930b28 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.256979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 17/33] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:50.257666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788fc1-fb3c-4e03-ba90-352516930b28 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:50.298185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:50.302734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a21ce-ea30-4e05-9fb0-93c4475f46db None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.339733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.340929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.341207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.341476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.342638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.344506 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.346108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 17/34] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:50.346849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:50.354147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:50.394141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788fc1-fb3c-4e03-ba90-352516930b28 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.437338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.438475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.438765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.439048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.440151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.442114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977ab32-f347-4406-95d5-088ed32ec8b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.467953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0b9e7-fa35-4ee8-992e-8b503eed19ea None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.503993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.504734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.505079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.505465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.506243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.507762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.601482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 18/35] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 07:44:50 2022] POST /identity/v3/roles => generated 219 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:44:50.603065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418c81-a588-42f4-a779-07213501e1a9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.603365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418c81-a588-42f4-a779-07213501e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.603648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418c81-a588-42f4-a779-07213501e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.628164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-03418c81-a588-42f4-a779-07213501e1a9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:50.629894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-03418c81-a588-42f4-a779-07213501e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:50.661772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 18/36] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:44:50 2022] POST /identity/v3/projects => generated 301 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:44:50.664002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:50.803896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 19/37] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:50.807055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ac99-0b70-42c9-b2fb-03406ff2671c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.807344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ac99-0b70-42c9-b2fb-03406ff2671c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.807626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898ac99-0b70-42c9-b2fb-03406ff2671c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.809936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 20/38] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:50.820246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f65765-3d32-4673-82f3-4c3631bb9b97 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.820562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f65765-3d32-4673-82f3-4c3631bb9b97 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.820823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f65765-3d32-4673-82f3-4c3631bb9b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.825995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:50.866303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e1f65765-3d32-4673-82f3-4c3631bb9b97 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:50.867553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e1f65765-3d32-4673-82f3-4c3631bb9b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:50.870018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:50.905796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acace59-218a-4bb0-a053-4df92c97259f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:50.948072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:50.948756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.949039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.949519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] PATH_INFO: `/v3/domains/default/users/d247b00da71b4a5ebefbb06a796143be/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.954268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d247b00da71b4a5ebefbb06a796143be, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:50.956111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:50.969833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:44:50.971836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecb91-3f64-4fcd-b568-d084014bc58b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.972358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecb91-3f64-4fcd-b568-d084014bc58b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.972956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ecb91-3f64-4fcd-b568-d084014bc58b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.974576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 22/40] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:50.981522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:50.982438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:50.982940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:50.984936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 19/41] 10.222.0.15 () {58 vars in 1361 bytes} [Tue Sep 13 07:44:50 2022] PUT /identity/v3/domains/default/users/d247b00da71b4a5ebefbb06a796143be/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:44:51.005598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:51.007357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:51.087457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 23/42] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:44:51.143611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:51.254846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0c24a-4440-4146-8158-259b3bc9fb37 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:51.269423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:51.271863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:51.299114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:51.299742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:51.300026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:51.300302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:51.301223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:51.303059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:51.325411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 20/43] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 07:44:51 2022] POST /identity/v3/roles => generated 225 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:44:51.361629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:51.375089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:51.424343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ff278-9d4a-44c2-a84e-0f1804144245 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:51.467505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:51.468222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:51.468508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:51.468780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:51.470337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:51.472051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:51.474010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 24/44] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:51.479961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:51.509651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777fa55-220c-4f83-89b8-0d190aed9e07 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:51.561079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:51.561713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:51.562309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:51.562587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:51.563344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:51.564751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad014d-6d86-419f-8b8e-ab197cbff99e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:51.585123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 21/45] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 07:44:51 2022] POST /identity/v3/roles => generated 223 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:44:51.604081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0ee4e-26c3-413d-af91-7544ff399802 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:51.648066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:51.648759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:51.649044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:51.649312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:51.650066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:51.651625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2d0c-30fe-475f-8722-4da7111e4916 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:51.658847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:51.685705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 25/46] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:44:51 2022] POST /identity/v3/projects => generated 312 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:44:53.235768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581cda1-70e1-44b7-a5f8-db6725a6a0de None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.236292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581cda1-70e1-44b7-a5f8-db6725a6a0de None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.236292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581cda1-70e1-44b7-a5f8-db6725a6a0de None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.237572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 22/47] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:53.242897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31cc71-2a52-4241-888e-2208c2c20ce3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.243311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31cc71-2a52-4241-888e-2208c2c20ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.243572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31cc71-2a52-4241-888e-2208c2c20ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.261312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5d31cc71-2a52-4241-888e-2208c2c20ce3 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:53.263093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5d31cc71-2a52-4241-888e-2208c2c20ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:53.326694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 26/48] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:44:53.595561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581cda1-70e1-44b7-a5f8-db6725a6a0de None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:53.735835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:53.807623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581cda1-70e1-44b7-a5f8-db6725a6a0de None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:53.841065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:53.841695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.842005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.842279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.852265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:44:53.856820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdab0-ed86-4f9b-8ef6-2de3a66b12da None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.857103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdab0-ed86-4f9b-8ef6-2de3a66b12da None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.857376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401cdab0-ed86-4f9b-8ef6-2de3a66b12da None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:44:53.858035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:44:53.858883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:44:53.859648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:44:53.860222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 27/49] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:44:53.860285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 23/50] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:44:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:44:53.863674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.863965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.864247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.865148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:53.884651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:53.884651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:53.954505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 28/51] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:53.955136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:53.962180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024ee0-c25d-4176-b4c1-732c888560a0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:53.995386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:53.995991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:53.996987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:53.997261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:53.997966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:53.998272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:53.999645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.007741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 24/52] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:44:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:54.013644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:54.062447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:54.111798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade08e0-6b75-4de4-a353-aec8dd8a62d6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:54.233746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:54.247908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:54.248553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:54.248832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:54.249100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:54.250478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:54.251935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.253488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 29/53] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:44:54 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:54.258872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:54.332497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4df99-c65b-4d01-93b9-31fa8361d434 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:54.356976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9c56-d3eb-4115-9739-2f90bac39646 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:54.366163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:54.366734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:54.367014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:54.367289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:54.368394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:54.369876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.371467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 25/54] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:54.373092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:54.377395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:54.391008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:54.391642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:54.391924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:54.392198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:54.393327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:54.394841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.396349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 30/55] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:54.402018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:54.402980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:54.478956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998b09-b787-4b10-a262-d67a27c592c7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:54.499943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc30130-602e-4425-b6f6-30916f5da296 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:54.511716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:54.512330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:54.512612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:54.512886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:54.513538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:54.515030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8519f-df4f-48e6-9509-f3fe7a0d9100 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.533199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:54.533942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:54.534226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:54.534497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:54.535288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.domain.id=default, user.id=d247b00da71b4a5ebefbb06a796143be) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:44:54.535627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:54.537116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32542c94-48c3-4956-bc37-853450e0c9b7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:54.541998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 26/56] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Sep 13 07:44:54 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:44:54.546324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 31/57] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Sep 13 07:44:54 2022] GET /identity/v3/role_assignments?user.id=d247b00da71b4a5ebefbb06a796143be&scope.domain.id=default&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:44:56.719459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9791fb-045e-461c-8e8d-54365bbe3dd0 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:56.719980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9791fb-045e-461c-8e8d-54365bbe3dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:56.720065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9791fb-045e-461c-8e8d-54365bbe3dd0 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:56.721350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 27/58] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:44:56.726566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:56.726949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:56.727113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:56.744763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:44:56.746472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:44:56.802368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:56.825918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 32/59] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:44:57.094598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9791fb-045e-461c-8e8d-54365bbe3dd0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:57.172098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:44:57.196090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9791fb-045e-461c-8e8d-54365bbe3dd0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:57.228997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:57.229625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:57.229941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:57.230219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:57.231901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:57.233349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7b685-ed63-484e-8d83-b16fac3b120e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:57.234944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 28/60] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:44:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:44:57.240558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:44:57.333266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d9f-7aea-42ef-8ce8-92a3b1ce543f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:44:57.365693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:44:57.366262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:44:57.366542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:44:57.366814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:44:57.367464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:44:57.368911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc2f42-e2a6-4b0e-8a7d-67cf31601a01 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:44:57.394154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 33/61] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:44:57 2022] POST /identity/v3/users => generated 275 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:00.690990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c317-f0f8-4b2b-b3d7-df4ca294a766 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:00.691497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c317-f0f8-4b2b-b3d7-df4ca294a766 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:00.691571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41c317-f0f8-4b2b-b3d7-df4ca294a766 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:00.693035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 29/62] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:00.704894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac9b22-934a-4e9d-9312-d069aa7f98b8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:00.705287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac9b22-934a-4e9d-9312-d069aa7f98b8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:00.705449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac9b22-934a-4e9d-9312-d069aa7f98b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:00.734660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-46ac9b22-934a-4e9d-9312-d069aa7f98b8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:00.735862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-46ac9b22-934a-4e9d-9312-d069aa7f98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:00.761931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858af8e-a34f-4167-83dc-841e0ffc288c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:00.762555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858af8e-a34f-4167-83dc-841e0ffc288c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:00.763029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858af8e-a34f-4167-83dc-841e0ffc288c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:00.765023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 30/63] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:00.771568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c6b24-232a-4718-9d27-62a2842de841 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:00.772090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c6b24-232a-4718-9d27-62a2842de841 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:00.772378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:00.773079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c6b24-232a-4718-9d27-62a2842de841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:00.804574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b09c6b24-232a-4718-9d27-62a2842de841 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:00.807080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b09c6b24-232a-4718-9d27-62a2842de841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:00.827863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 34/64] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:00.915940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 31/65] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:01.016895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a241f0-3c99-4a1e-a938-10118ad1dc4e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.017491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a241f0-3c99-4a1e-a938-10118ad1dc4e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.018006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a241f0-3c99-4a1e-a938-10118ad1dc4e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.019548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 35/66] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:01.025008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb88dc-038e-4be9-a52e-5cf16e476abb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.025303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb88dc-038e-4be9-a52e-5cf16e476abb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.025586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb88dc-038e-4be9-a52e-5cf16e476abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.055050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-18bb88dc-038e-4be9-a52e-5cf16e476abb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:01.056306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-18bb88dc-038e-4be9-a52e-5cf16e476abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:01.141434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a241f0-3c99-4a1e-a938-10118ad1dc4e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.150273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 32/67] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:01.169862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de757-34de-41d8-9aeb-e4ec7e267da1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.170235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de757-34de-41d8-9aeb-e4ec7e267da1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.170454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de757-34de-41d8-9aeb-e4ec7e267da1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.171981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 33/68] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:01.177095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.177385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.177671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.190051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.211491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:01.212685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:01.301596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 34/69] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:01.302911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.326145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a241f0-3c99-4a1e-a938-10118ad1dc4e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.393059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.394246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.395939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.397180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.419249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:01.425821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:01.426739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:01.427502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 36/70] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:01.428127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.465301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.469027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e54904-b906-44d2-8b0b-b1498a94b79e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.514149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.516217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.516712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.517209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.519373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.532440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:01.538158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:01.538994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:01.539764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 35/71] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:01.540399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.558341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2da01a-84d7-4429-a566-c176c1d214cd None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.593897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.594533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.594814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.595088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.605166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:01.610406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:01.611240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:01.611993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:01.616467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 37/72] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:01.616905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.646571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3e078-25a0-46ed-9bc7-1b07947fcbe0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.681095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.681712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.682021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.682295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.682966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:01.683784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:01.685186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156334b-8e41-4fcd-b168-d898fb08514a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:01.693291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 36/73] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:01.694486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1846-246f-4a2a-a2c7-25998d9fd739 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.694773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1846-246f-4a2a-a2c7-25998d9fd739 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.695054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1846-246f-4a2a-a2c7-25998d9fd739 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.696423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 37/74] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:01.696978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb1846-246f-4a2a-a2c7-25998d9fd739 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.704460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.717385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740653e-af62-412f-8eb8-a6b4f14f3c52 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.750149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.750762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.751047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.751321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.751989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:01.752298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:01.753460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.753707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:01.766053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:01.771595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.798621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb1846-246f-4a2a-a2c7-25998d9fd739 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.833856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.834467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.834752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.835057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.835729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:01.836035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:01.837435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:01.846401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 38/76] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:01.847277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:01.872188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550032b-a84b-4a2b-9e44-b33ca7e713ff None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.908389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.909021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.909297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.909564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.919331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-eb82fdaf-7ce7-4fe5-80ca-cb6f07dd79ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:01.924072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:01.924935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:01.925803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 39/77] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:01.926441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.927109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.927109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.939759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.945628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:01.946865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:01.947236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a3e91-cd53-4375-880e-8f8e9decdf96 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:01.964651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:01.981574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:01.982235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:01.982562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:01.982818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:01.996499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:01.997974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:01.999572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 39/78] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.000331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.016469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 40/79] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:02.017300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.112793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070243bc-b8c3-4045-a1b0-6d7bced1508e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.124841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af97362-08ee-459e-b201-14d1f4985925 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.148073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.148703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.148982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.149253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.150425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.151865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.153636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 40/80] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:01 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.154233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.165276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.166308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.166858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.167273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.169066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.171362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.173476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.174576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.182379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.210028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.265235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d36ec-0218-4822-b8e2-672b27161bc7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.288148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96974ab0-c0c5-443c-a5d8-1f2bafc13a33 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.299536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.300147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.300426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.300703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.301364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:02.301665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.303071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.311899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.312683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.327704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.328314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.328594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.328868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.395988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.397460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.400051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.412689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcce1d7-7851-41fe-badf-3240e134b80c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.419959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.447869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.448483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.448762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.449064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.495942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 42/83] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:02.496771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.514512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.515988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.593848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dafa3a-af2a-4d54-8914-4d4b522e36b9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.596088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.612253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 42/84] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:02.613087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.621744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.629675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.630319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.630597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.630868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.697867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.699332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.711076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9255dd-61d6-4677-8de9-c0b572a41df9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.745422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.746102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.746383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.746662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.747770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.749216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.750921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.751683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.796775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 43/86] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:02.797955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.801002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.827306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:02.854023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0257-c72f-4609-9ef6-cbc3fafd789b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.889542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.890213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.890499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.890777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.891943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.893400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.899469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.899469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.914457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237347c-18b6-435c-b2dc-4c53649870ca None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:02.950031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:02.950599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:02.950880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:02.951157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:02.952279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:02.953765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:02.955414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:02.956140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:02.999819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7631ee-bc1b-468d-a7e5-877637979a52 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.023490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.033659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.034320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.034600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.034874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.036004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.037444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.039110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 45/89] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:02 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.042576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.053348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc52f3-a9eb-44c9-8238-2b2f1ea2b1fe None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.054025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.087564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.088176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.088460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.088734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.089396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.091426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.111861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 45/90] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:02 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:03.112682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.160184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51255ca-52cf-4fa6-bd4e-ae15392b9bee None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.202471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.203186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.203476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.203757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.204977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.206809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.213652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.213652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.232027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233265fb-b2d8-422c-b476-2d5a6547f1c7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.262869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.272554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.273205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.273777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.274126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.306366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.337337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae1d5f-a479-4169-b94c-3ee739eac4a2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.355214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.356861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.378998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.379896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.380177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.380455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.381197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:03.381502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.382997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.391027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 47/92] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.393450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.502410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 46/93] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:03.503468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.534393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.543873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b2234-a6d1-42cd-b60f-b0b0b26fbfc9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.587370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.588239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.588693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.589112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.590169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=d247b00da71b4a5ebefbb06a796143be) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:03.590650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.592262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.598633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/role_assignments?user.id=d247b00da71b4a5ebefbb06a796143be&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.599643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.619557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.633822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bcc90-e14e-4094-9d77-48c5261807de None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.682630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.683826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.684117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.684394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.685155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:03.689533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.694790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.706148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 47/95] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=b1177515548b419b92d3359f27435e02 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.773890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8416314-ec8a-4dfe-a421-5e400a4ff82b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:03.819884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:03.821683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:03.821683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:03.821683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] PATH_INFO: `/v3/projects/8df02a8dfd594d218c8085ed8cd194a8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:03.822646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] RBAC: Authorizing `identity:get_project(project_id=8df02a8dfd594d218c8085ed8cd194a8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:03.824422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2f6bf-8fc4-443a-8484-2df2964389c5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:03.826463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/projects/8df02a8dfd594d218c8085ed8cd194a8 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:03.833628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:03.968033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:03.979434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fe380-8d63-42aa-a3b4-7c7d786174f6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:04.022864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:04.023552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:04.023849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:04.024185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:04.024877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=8df02a8dfd594d218c8085ed8cd194a8, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:04.025191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:04.026703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec45e4f-2696-4446-8b4a-f95504da4e62 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:04.038547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:03 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=8df02a8dfd594d218c8085ed8cd194a8&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:06.253164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274bb32-a938-471c-83df-8572c40fa0d5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.254220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274bb32-a938-471c-83df-8572c40fa0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.254733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274bb32-a938-471c-83df-8572c40fa0d5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.256403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:06.262871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44703c-aba4-4e60-9209-7733b4011a69 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.263802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44703c-aba4-4e60-9209-7733b4011a69 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.264284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44703c-aba4-4e60-9209-7733b4011a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.290491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9f44703c-aba4-4e60-9209-7733b4011a69 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:06.292036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9f44703c-aba4-4e60-9209-7733b4011a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:06.382718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 49/99] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:06.412663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587eef2-0aa4-4b10-b491-e212064aa6c7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.413266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587eef2-0aa4-4b10-b491-e212064aa6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.413766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587eef2-0aa4-4b10-b491-e212064aa6c7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.415319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 51/100] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:06.420689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7b8c-f300-444a-90ed-07529848deb5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.420980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7b8c-f300-444a-90ed-07529848deb5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.421259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7b8c-f300-444a-90ed-07529848deb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.446798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8e0b7b8c-f300-444a-90ed-07529848deb5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:06.448012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e0b7b8c-f300-444a-90ed-07529848deb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:06.528369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 50/101] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:06.795185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01c58-9bbe-4315-a342-c11256899d4e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.795879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01c58-9bbe-4315-a342-c11256899d4e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.796438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf01c58-9bbe-4315-a342-c11256899d4e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.798030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 52/102] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:06.803604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.803899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.804182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.827705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:06.828894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:06.842566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:06.892462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a344-cbd9-438c-af89-8ca03abeae90 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.893008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a344-cbd9-438c-af89-8ca03abeae90 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.893492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26a344-cbd9-438c-af89-8ca03abeae90 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.900181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 53/103] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:06.906699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:06.907243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:06.907746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:06.925474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 51/104] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:06.927631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:06.933767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:06.944142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.064360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 54/105] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:07.066377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.072569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.135781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8c758-4113-48ae-9e8b-9f1be5ccb3b9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.196058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.196726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.197013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.197284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.220091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4f52fdc1-4e5e-4ab1-b109-c069aae32104 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:07.225023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:07.226136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:07.226897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:07.227562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 52/106] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:07.230926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad71144-4605-4953-a20b-5fcfd1932a35 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.231218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad71144-4605-4953-a20b-5fcfd1932a35 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.231496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad71144-4605-4953-a20b-5fcfd1932a35 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.233798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 53/107] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:07.236638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad71144-4605-4953-a20b-5fcfd1932a35 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.247106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766fc6e-061c-4db0-90b9-33a46cf911cf None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.255562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.289900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.291647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.292108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.293094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.308665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:07.316172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:07.317254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:07.318257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:07.321598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 55/108] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:07.323001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.360340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad71144-4605-4953-a20b-5fcfd1932a35 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.402501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.403303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.403681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.404065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.404837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:07.405213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.406715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2c0a9-7eec-4cba-a48e-4975b7f2a04a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.408591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.417560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 54/109] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.419088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.419465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.419808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.439333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:07.440519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:07.454427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9d5-f1e5-40b3-96cb-5ac0a070a61a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.489941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.490573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.490854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.491128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.501232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:07.502898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:07.506394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:07.507242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:07.508017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 56/110] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:07.510499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 55/111] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:07.511813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.513534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.620480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc05490-a33d-4a96-88d9-05645260c869 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.624642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7515431-9cd0-4c16-9851-2ad11b492024 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.646309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.659965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.660232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.660587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.660874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.660968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.661143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.661233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.661417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] PATH_INFO: `/v3/users/7abfb1e92ecc4d469fd2338c39a75f99` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.661850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:07.662162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.663561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819184c-1f1e-4b06-a9fc-9896b56e0ded None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.672019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 57/112] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.676648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b8986-39d9-446d-8e8b-0aaa2bea0c49 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.676648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b8986-39d9-446d-8e8b-0aaa2bea0c49 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.676648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b8986-39d9-446d-8e8b-0aaa2bea0c49 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.676648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 58/113] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:07.676648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b8986-39d9-446d-8e8b-0aaa2bea0c49 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.684468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] RBAC: Authorizing `identity:get_user(user_id=7abfb1e92ecc4d469fd2338c39a75f99)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.685958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.687942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 56/114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/users/7abfb1e92ecc4d469fd2338c39a75f99 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.688709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.748047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.788458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b8986-39d9-446d-8e8b-0aaa2bea0c49 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.791875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3818ed-a2da-43ff-ad69-8b73c93ca465 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.821379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.822037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.822317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.822591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.824311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.825864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.826187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.826800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.827081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.827356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.827483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 59/115] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.828029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:07.828331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.829694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22bbd2-b567-4e22-8bd3-d4ad8bca06e7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.832698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.838028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 57/116] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.839344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.839632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.839911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.857645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:07.858887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:07.889279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.937769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 58/117] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:07.937769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:07.938019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6456e6-83b3-47e2-960d-b59d761afa63 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:07.973513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:07.974172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:07.974459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:07.974730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:07.975855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:07.976161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:07.978502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:07.980576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 60/118] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:07.981483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.032368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16361f-196b-4522-b742-55906e8d3631 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.069465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.070205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.070488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.070761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.090084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d3b88-9340-40da-956a-12891d3783a4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.116580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.130658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.131291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.131572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.131852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.141034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.142541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.143098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:08.148439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:08.149461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:08.150421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:08.153780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 61/119] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:08.153780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.199013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.250987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 59/120] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:08.253441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6207b5-1116-486e-8814-bdafec0c5458 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.255743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.286066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.286714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.286996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.287273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.287930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.289395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.317482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 62/121] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:45:08 2022] POST /identity/v3/projects => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:08.318341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.358562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.364725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463e50-fb17-4ef1-82c4-81279f61a935 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.407940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.408519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.408790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.409067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.410390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.411793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.413659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 60/122] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:08.419565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.438813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e055045-f87a-46bc-b450-aee61e35b244 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.439572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.477656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.478348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.478633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.478913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.480797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.482316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.484011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 63/123] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:08.488828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.539696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae97cd-1cfa-449d-9621-070b00630c79 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.576742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.577404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.577681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.578023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.578713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:08.579015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.580389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.588860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 61/124] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:08.590217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.608670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e208fd-a588-4f0a-9c86-4725061ff7c0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.610789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.647556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.648202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.648489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.648766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.660591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:08.665615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:08.666528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:08.668388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 64/125] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:08.669271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.675826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.703304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076cd38-50c9-41e3-ae04-d5ec251d4b77 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.741303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.742002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.742280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.742556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.743697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.745155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.747015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 62/126] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:08.747800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.785055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26676a91-9a11-4209-a0c6-945de10f6502 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.819890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.820682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.821061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.821431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.822695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.824742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.826800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 65/127] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:08.828657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.858995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.865423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149bf21-6e13-45ba-b73d-3046d05ba5ac None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:08.904213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:08.904984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:08.905337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:08.905687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:08.907750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, user_id=7c91ef32acfb46029d0aa8660b4c5b18, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:08.909297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:08.925248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 63/128] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:08 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:08.926151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:08.943466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:08.958149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a91b78-43c0-4710-89b5-4897bed02622 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.006008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.006922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.007679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.008099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.010120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.012711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.015954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 66/129] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:09.017211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.065434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194584c6-b81f-4885-b6be-05bac40ef620 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.105745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.106585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.107073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.107355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.108127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:09.108553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.110115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.123778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 64/130] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:09.124724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.156556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:09.162582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd86a-6f33-46ec-b8da-9de8c33a16d8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.203035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:09.211292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.211977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.212262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.212542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.213291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7abfb1e92ecc4d469fd2338c39a75f99) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:09.213606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.215734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.223414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 67/131] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:09 2022] GET /identity/v3/role_assignments?user.id=7abfb1e92ecc4d469fd2338c39a75f99&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:09.224197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.270039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e10ab-1cf8-4c12-86b6-444acd00b9df None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.326038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.326736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.327023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.327307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/d247b00da71b4a5ebefbb06a796143be/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.331462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, user_id=d247b00da71b4a5ebefbb06a796143be, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.332968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.369265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 65/132] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:09 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/d247b00da71b4a5ebefbb06a796143be/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:09.369934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.423547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a94a9-cf68-48e8-b564-753c9d54573d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.469667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.470627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.470912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.471190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] PATH_INFO: `/v3/projects/8df02a8dfd594d218c8085ed8cd194a8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.473648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] RBAC: Authorizing `identity:get_project(project_id=8df02a8dfd594d218c8085ed8cd194a8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.475168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.477778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 68/133] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:09 2022] GET /identity/v3/projects/8df02a8dfd594d218c8085ed8cd194a8 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:09.483938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.540183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:09.557614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f189a-7f68-4c2a-9970-c3ed5b9dbf16 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.611616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:09.627239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.627890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.628171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.628449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.629602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.631151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.638323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 66/134] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:09 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:09.647029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.682034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec384562-28d6-4e5c-a587-a723d2f1b546 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.740572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.741605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.741924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.742348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] PATH_INFO: `/v3/projects/8df02a8dfd594d218c8085ed8cd194a8/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.747768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] RBAC: Authorizing `identity:create_grant(project_id=8df02a8dfd594d218c8085ed8cd194a8, user_id=7c91ef32acfb46029d0aa8660b4c5b18, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.749258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.777600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 69/135] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:09 2022] PUT /identity/v3/projects/8df02a8dfd594d218c8085ed8cd194a8/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:09.805078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b473b7-8ff5-404e-b169-e44acaca88c3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.842774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:09.843427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:09.843706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:09.843978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:09.845116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:09.846721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0db50-18d2-4d61-8973-89ddf42755b4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:09.848524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 67/136] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:09 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:09.854483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:09.982294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecf26a-00c1-4505-96d5-278ab9ee3818 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:09.984260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:10.033719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:10.034357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:10.034638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:10.034916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/b1177515548b419b92d3359f27435e02` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:10.037167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, user_id=7c91ef32acfb46029d0aa8660b4c5b18, role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:10.038778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17431245-9351-4079-a3fc-308aeae4e286 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:10.066345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 70/137] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:09 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7c91ef32acfb46029d0aa8660b4c5b18/roles/b1177515548b419b92d3359f27435e02 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:10.934390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038adbe-6095-4446-bf0d-e124a18dc5cd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:10.934929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038adbe-6095-4446-bf0d-e124a18dc5cd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:10.934929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038adbe-6095-4446-bf0d-e124a18dc5cd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:10.936725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 68/138] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:10.947486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:10.947936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:10.948031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:10.967316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:10.968500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:11.055791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 71/139] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:11.515645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038adbe-6095-4446-bf0d-e124a18dc5cd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:11.596354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:11.651005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038adbe-6095-4446-bf0d-e124a18dc5cd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:11.699198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:11.699857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:11.700150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:11.700429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:11.702500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:11.704222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763866-443d-4ce6-89e1-a62e4696dc3c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:11.706550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 69/140] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:11.711739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:11.817899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:11.840081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b7aa3-292a-453e-a9b1-cb57463f52af None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:11.875776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:11.876432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:11.876712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:11.876981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:11.877649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:11.879116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0c252-bef0-4737-aafe-de11965f0801 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:11.909326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 72/141] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:11 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:12.548171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce0ee-6ad0-44c8-9a00-2f25e10708aa None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.548707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce0ee-6ad0-44c8-9a00-2f25e10708aa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.548707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dce0ee-6ad0-44c8-9a00-2f25e10708aa None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.550240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 70/142] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:12.555682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a985-e3ef-4126-9f11-a78e0841c38a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.556481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a985-e3ef-4126-9f11-a78e0841c38a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.556962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a985-e3ef-4126-9f11-a78e0841c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.583226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1c31a985-e3ef-4126-9f11-a78e0841c38a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:12.584852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1c31a985-e3ef-4126-9f11-a78e0841c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:12.663213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 73/143] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:12.726288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca0f-cd3b-4206-8a83-ddea28a71fea None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.726680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca0f-cd3b-4206-8a83-ddea28a71fea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.726862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca0f-cd3b-4206-8a83-ddea28a71fea None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.728598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 71/144] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:12.734738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95613db-2aef-4e82-8b6f-bbfba2be7409 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.735351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95613db-2aef-4e82-8b6f-bbfba2be7409 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.735814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95613db-2aef-4e82-8b6f-bbfba2be7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.759735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d95613db-2aef-4e82-8b6f-bbfba2be7409 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:12.761554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387af21-f967-4f93-8c31-0ae5019c32d6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.761886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387af21-f967-4f93-8c31-0ae5019c32d6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.762165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387af21-f967-4f93-8c31-0ae5019c32d6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.762484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d95613db-2aef-4e82-8b6f-bbfba2be7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:12.763730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 72/145] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:12.768957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.769240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.769522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.811038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:12.812553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:12.845639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:12.848737 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 74/146] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:12.851089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0416-6dc0-4cd2-8de6-1beba29119a0 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.851363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0416-6dc0-4cd2-8de6-1beba29119a0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.851736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0416-6dc0-4cd2-8de6-1beba29119a0 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.853239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 75/147] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:12.859444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:12.859444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:12.859748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:12.884904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:12.887089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:12.961120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 73/148] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:12.963322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 76/149] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:13.058511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.129179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:13.175063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e34ae-42e6-454d-b036-8b4af05fc284 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.214892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.215540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.215819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.216093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.232333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:13.237194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:13.238092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:13.238829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 74/150] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:13.246285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.293909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.379997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19625415-671e-4302-8e5d-826cd3989d86 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.418605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:13.431636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.432264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.432546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.433378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.434098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:13.434408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:13.436243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a2140-1122-4c38-8ee5-8dcf28503255 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:13.445875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 77/151] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:13.447074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc80f0-2566-49eb-85d5-d7449cac03d3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.447593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc80f0-2566-49eb-85d5-d7449cac03d3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.447943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc80f0-2566-49eb-85d5-d7449cac03d3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.449296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 78/152] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:13.450144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc80f0-2566-49eb-85d5-d7449cac03d3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.459200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2db0-73d8-409a-b237-26ffef8cd99e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.487379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:13.504675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.505321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.505599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.505928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.523138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:13.529290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:13.530305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:13.531176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 75/153] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:13.531808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.584140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc80f0-2566-49eb-85d5-d7449cac03d3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.627456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.628101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.628657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.628932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.642930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:13.649051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:13.649918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:13.650664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 79/154] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:13.651300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.669894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b6993-5d1e-486f-a5b9-5f48aa3023e7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.717022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:13.724190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.725280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.725561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.725939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.740382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7b675b40-56ea-4d6c-9971-ac6cf73b7b4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:13.745568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:13.746790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:13.747748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 76/155] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:13.748540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.748540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.748540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.767954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:13.769138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:13.783001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3baf04-20c4-410a-97a5-c470d1023b3c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.812048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:13.820527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.821374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.821768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.822143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] PATH_INFO: `/v3/users/d247b00da71b4a5ebefbb06a796143be` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.823583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] RBAC: Authorizing `identity:get_user(user_id=d247b00da71b4a5ebefbb06a796143be)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:13.825314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:13.827495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 80/156] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/users/d247b00da71b4a5ebefbb06a796143be => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:13.828488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.848502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 77/157] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:13.849594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:13.955625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b680f-add2-4c11-80dd-83f2f0aaf37e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:13.991433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:13.992150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:13.992437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:13.992709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:13.993487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:13.993829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:13.995240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.004044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 78/158] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.005024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.015818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747294b6-aaf1-4c34-a7db-3ffdb5c2105e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.040836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.043546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.075501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.076124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.076452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.076730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.077511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:14.077838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.079371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.093420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 81/159] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.094388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.114597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b72387-c917-482d-a958-715fb86e7c26 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.150556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.151250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.152265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.152696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.153513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:14.153987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.155532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.164745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 79/160] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.165864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.266808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df500b-4a7b-4736-8b4e-ce45cf82c7f7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.313714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.321078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852d7e2-cf67-40b4-8598-af1ef44a2655 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.362595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.363222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.363501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.363772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.384266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.385299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.386341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.386843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] PATH_INFO: `/v3/users/7abfb1e92ecc4d469fd2338c39a75f99` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.388358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] RBAC: Authorizing `identity:get_user(user_id=7abfb1e92ecc4d469fd2338c39a75f99)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.391046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.394217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 80/161] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/users/7abfb1e92ecc4d469fd2338c39a75f99 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.395494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.471232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.521521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.522938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.534444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6070bc5-ddd5-4c1e-bdb7-fd5a23a33789 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.572694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.573897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.574698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.574980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.576124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.577692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.579604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 81/162] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.580633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.615528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.694044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2c7-8c22-47a5-b73f-7e0bebdebe09 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.733101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.733759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.734082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.734359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.735487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.737013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.738675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 82/163] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.740025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.788169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 82/164] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:14.788800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:14.850557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dca951-a2f5-42dd-b3ec-b9d0a03ef2b3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:14.854029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:14.889088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:14.889977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:14.890267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:14.890551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:14.891741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:14.893568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:14.895240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 83/165] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:14.896017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.003541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.012630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccadf19-ab4a-4015-ad82-4ccb22247d34 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.048708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.049390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.049713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.050028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.066616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a65f7-f676-43a0-b1c4-42a3078549eb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.089242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.124951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:15.126479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:15.178455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.179104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.179383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.179653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.207575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:15.224704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:15.225640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 83/166] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:15.226639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.239428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 84/167] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:15.240316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.319411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.350618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942fb30-428f-4f37-868a-fb0d9b736521 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.409010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bca54-997d-4d9e-b733-badd48081265 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.423067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.423729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.424011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.424284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.425454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:15.426947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:15.428897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 85/168] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:15 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:15.429653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.449486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.450140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.450415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.450691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.466903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.557627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:15.559259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:15.672793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf0755-efc6-4f8b-8cd9-daefdeb286ea None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.690485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 84/169] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:15.691360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.735263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.774593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.775274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.775552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.775827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7abfb1e92ecc4d469fd2338c39a75f99/roles/17eb64f9d07f42c38e67fd8bbc093a79` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.783168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, user_id=7abfb1e92ecc4d469fd2338c39a75f99, role_id=17eb64f9d07f42c38e67fd8bbc093a79)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:15.784742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:15.789490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.798191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056e8f4-6dfc-455e-a1dd-87e84f2ef63b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:15.812783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 86/170] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:15 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/users/7abfb1e92ecc4d469fd2338c39a75f99/roles/17eb64f9d07f42c38e67fd8bbc093a79 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:15.817954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.839563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:15.840272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:15.840553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:15.840823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:15.841606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:15.841940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:15.843420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:15.853382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 85/171] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:15.854104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:15.964987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:15.967859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a3678-1879-4f49-84e8-7e14fd73aa4d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.003559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018553-03f6-47e5-ae32-62493a416c48 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.007342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.007963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.008465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.008875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.009783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=d247b00da71b4a5ebefbb06a796143be) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:16.010231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.012057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.019457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 86/172] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:15 2022] GET /identity/v3/role_assignments?user.id=d247b00da71b4a5ebefbb06a796143be&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.025772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.054042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.054883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.055168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.057135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.060625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.062682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b8ba1-5688-4303-b2ce-edfa5feba6ee None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.067443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 87/173] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:15 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.068788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea19dd-63fc-4e05-9e77-188825ae7f5e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.069069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea19dd-63fc-4e05-9e77-188825ae7f5e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.069343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea19dd-63fc-4e05-9e77-188825ae7f5e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.076959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 88/174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:16.079769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea19dd-63fc-4e05-9e77-188825ae7f5e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.170320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:16.188342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38c938-95eb-4c7a-98d8-786e1dd935fd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.210065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea19dd-63fc-4e05-9e77-188825ae7f5e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.246741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.247369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.247650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.247921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.250689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.252875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540fb16-7089-484a-9fc7-4ccfae7a8e5e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.254886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 87/175] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.255300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.256206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.256321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4275346-3b93-4513-b5c6-4f366ebcef02 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.256520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4275346-3b93-4513-b5c6-4f366ebcef02 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.256793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4275346-3b93-4513-b5c6-4f366ebcef02 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.257200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.257654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] PATH_INFO: `/v3/users/7c91ef32acfb46029d0aa8660b4c5b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.258188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 88/176] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:16.260536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] RBAC: Authorizing `identity:get_user(user_id=7c91ef32acfb46029d0aa8660b4c5b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.262360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17820b53-7d64-4e3f-8571-19f52c501b0e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.262482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4275346-3b93-4513-b5c6-4f366ebcef02 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.264502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 89/177] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/users/7c91ef32acfb46029d0aa8660b4c5b18 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.266182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.266660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.268081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.294681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:16.296290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:16.302906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:16.388499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 90/178] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:16.389709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.420359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4275346-3b93-4513-b5c6-4f366ebcef02 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.479781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.480783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.481273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.481798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.482745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:16.483256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.484926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78f89d-e670-4c5c-aeb4-0593af142c3c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.490123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:16.496858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 89/179] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=b1177515548b419b92d3359f27435e02 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.498404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903408-d941-4583-a4bc-d0299c32bb95 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.498881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903408-d941-4583-a4bc-d0299c32bb95 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.499356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903408-d941-4583-a4bc-d0299c32bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.520661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1c903408-d941-4583-a4bc-d0299c32bb95 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:16.522318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1c903408-d941-4583-a4bc-d0299c32bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:16.560413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0833b-9908-491d-bc89-b3bdd02fae20 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.621215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.623925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.624232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.624510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.625288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:16.625602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.627389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.634479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 91/180] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:16.635468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.731169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 90/181] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:16.732488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0037a3-714c-4291-b3b8-f1435ac22c62 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.732778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0037a3-714c-4291-b3b8-f1435ac22c62 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.733052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0037a3-714c-4291-b3b8-f1435ac22c62 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.734435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 91/182] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:16.735264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0037a3-714c-4291-b3b8-f1435ac22c62 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:16.773601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1a4e6-c26d-4e6c-8a40-d33a3dd9aaa6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:16.801528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:16.813280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:16.836299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:16.836970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:16.837251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:16.837527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:16.925413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:16.927010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407690c0-0855-4203-8b55-c6dd7e717494 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:16.998446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0037a3-714c-4291-b3b8-f1435ac22c62 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.070032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 92/183] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:17.071310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.071590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.077765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.084609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.085762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.085762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.085923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.111984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:17.112308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:17.114521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:17.123249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:17.124161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:17.124919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:17.125503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 92/184] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:17.126130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.149136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.208997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 93/185] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:17.210372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.327525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb47be-11da-4188-8d37-e5164086517d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.338520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.373760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.373760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.373760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.373760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.381754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37fd20-cbcd-46bc-a6b0-8ea12f604efd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.397321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:17.407312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:17.408262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:17.409033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 94/186] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:17.409667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.447872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.449703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.450439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.450717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.450991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] PATH_INFO: `/v3/projects/8df02a8dfd594d218c8085ed8cd194a8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.455169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] RBAC: Authorizing `identity:get_project(project_id=8df02a8dfd594d218c8085ed8cd194a8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.456493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.458189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 93/187] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/projects/8df02a8dfd594d218c8085ed8cd194a8 => generated 312 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.458961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.559953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799b80-059d-4c26-a194-289caa253b7a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.574853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4dd5-29d9-42c2-b1ab-249e3eb136f2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.597018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.597662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.597972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.598245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.598909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:17.599210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.600606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.609525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 95/188] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.610444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.617747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.618375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.618658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.618931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.619619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:17.619953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.621331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.629849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 94/189] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.630650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.648745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.702241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.726419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46361-7e1c-4507-a9ab-2a5db9c7acb8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.737415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21502b-35d7-4138-8268-b8c7810fa187 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.763407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.764132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.764412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.764683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.765414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=8df02a8dfd594d218c8085ed8cd194a8, user.id=7c91ef32acfb46029d0aa8660b4c5b18) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:17.765725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.767429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.771421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.772054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.772329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.772598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.782017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 96/190] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/role_assignments?user.id=7c91ef32acfb46029d0aa8660b4c5b18&scope.project.id=8df02a8dfd594d218c8085ed8cd194a8&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.782223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.782304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.782383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.782383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 95/191] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.782383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.902110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.903745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1b882-3923-4fd4-98da-706d0aa0039f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.912978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0ade3-8be5-4f11-b0e1-5b4f5ef40672 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:17.943490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.944139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.944417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.944688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.955557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:17.956597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:17.957276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:17.957910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:17.958689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:17.961651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.963590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.970288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 97/192] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.971139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:17.977940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:17.980139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:17.982110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 96/193] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:17.983103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:18.097042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5ca89-170e-41d6-bde1-999eab1b6d1d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.106873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e980a2e-70b4-4f42-9e64-5ef3128c344c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.132637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.133254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.133536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.133840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.145461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.145948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.147269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.147922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.148254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.148924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.150429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.174883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 98/194] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:17 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:18.175507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:18.234625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.236138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.257454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.324489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2662864-ac88-4639-b348-d7c652678e16 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.363559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 97/195] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:18.369096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:18.369839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.370599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.370837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.371226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.415694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.450324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.451809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ca6ba-a213-4544-9ce9-569cbe370b8d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.479045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67624deb-5824-40b2-bdfe-68f700d9cfc6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.518659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.531913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.531913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.532204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.532325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.533468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.535002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdaf06-ec0b-48b5-be35-386a2db85361 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.536679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 98/196] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:18 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:18.538856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d644-48e0-41e1-a233-6d075fc2c8f5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.539144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d644-48e0-41e1-a233-6d075fc2c8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.539415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d644-48e0-41e1-a233-6d075fc2c8f5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.540928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 99/197] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:18.544285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d644-48e0-41e1-a233-6d075fc2c8f5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:18.571121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 99/198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:18.572741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.573552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.573948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.595531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:18.596734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:18.677864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 100/199] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:18.678870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:18.683298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d644-48e0-41e1-a233-6d075fc2c8f5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.698951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.748518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.749138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.749421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.749693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.752457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=80629860440c468aa06d38511fc80e20) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:18.752784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.754285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.763731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 100/200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:18 2022] GET /identity/v3/role_assignments?user.id=80629860440c468aa06d38511fc80e20&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:18.824726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:18.843131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb176183-81f7-496d-a05e-e0c974da1901 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:18.881536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:18.882228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:18.882509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:18.882828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:18.886028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:18.889298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:18.889429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 101/201] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:18 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:18.900070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:19.025254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ebd9e-c417-497a-a428-154d51536088 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:19.051166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:19.066017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:19.066637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:19.066918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:19.067194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:19.067935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=80629860440c468aa06d38511fc80e20) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:19.068245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:19.073755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:19.077156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 101/202] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:18 2022] GET /identity/v3/role_assignments?user.id=80629860440c468aa06d38511fc80e20&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=b1177515548b419b92d3359f27435e02 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:19.246481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9034-0083-4035-98a0-cde6a172d3de None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:19.284543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:19.285428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:19.285981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:19.286396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:19.298983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:19.303971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:19.305037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:19.306025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 102/203] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:19.316135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:19.379748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:19.465479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e253-1fd8-46ee-9334-9d1f4d970d60 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:19.500783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:19.501801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:19.502285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:19.502757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:19.503631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:19.504155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:19.505762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:19.515097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 102/204] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:19.521622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:19.550327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:19.654519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8af2-a47a-4c46-9dc6-ac5cfe8347c4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:19.695390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:19.696361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:19.696840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:19.697341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] PATH_INFO: `/v3/users/7abfb1e92ecc4d469fd2338c39a75f99` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:19.698788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] RBAC: Authorizing `identity:get_user(user_id=7abfb1e92ecc4d469fd2338c39a75f99)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:19.700504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:19.702355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 103/205] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:19 2022] GET /identity/v3/users/7abfb1e92ecc4d469fd2338c39a75f99 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:19.709225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:19.843944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:19.847720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce90de-bd7a-44e8-bd36-0401f760807b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:19.904454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:19.905082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:19.905361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:19.905628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:20.015885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:20.017389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:20.149851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:20.153342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 103/206] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:20.159444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:20.280661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:20.310907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873601cb-9587-45a5-ad32-73e46d0f719a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:20.358452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:20.359094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:20.359374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:20.359648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:20.360823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:20.362327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ca1a1-9a31-4332-9120-bbbc2f1de2d8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:20.364274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 104/207] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:20 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:20.370421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:20.533650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e32f-e3c2-48c5-a94e-0750d4928778 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:20.579466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:20.580513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:20.580980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:20.581436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:20.582399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f, user.id=7abfb1e92ecc4d469fd2338c39a75f99) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:20.582899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:20.584605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbae79-b7a1-4770-a4d0-a0cfccc83e6d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:20.593878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 104/208] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:20 2022] GET /identity/v3/role_assignments?user.id=7abfb1e92ecc4d469fd2338c39a75f99&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:21.020303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ce845-aea3-4d27-964b-b856024ce038 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.020648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ce845-aea3-4d27-964b-b856024ce038 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.020856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ce845-aea3-4d27-964b-b856024ce038 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.022518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 105/209] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:21.028286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd56c4-c842-45b5-b50f-a061ba29128d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.028780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd56c4-c842-45b5-b50f-a061ba29128d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.029259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd56c4-c842-45b5-b50f-a061ba29128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.047822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-49fd56c4-c842-45b5-b50f-a061ba29128d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:21.049317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-49fd56c4-c842-45b5-b50f-a061ba29128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:21.115711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:21.126004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 105/210] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:21.222393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250ebbc-0768-418f-bac8-611df0462a8d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.222869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250ebbc-0768-418f-bac8-611df0462a8d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.222869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250ebbc-0768-418f-bac8-611df0462a8d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.224452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 106/211] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:21.229726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a6f7-8e79-40f3-a193-96882156eb15 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.230189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a6f7-8e79-40f3-a193-96882156eb15 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.230288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a6f7-8e79-40f3-a193-96882156eb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.250869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f0e1a6f7-8e79-40f3-a193-96882156eb15 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:21.252099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f0e1a6f7-8e79-40f3-a193-96882156eb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:21.335876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 106/212] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:21.380392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9256c-e580-4c9f-8287-6e0d26560c18 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.380674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9256c-e580-4c9f-8287-6e0d26560c18 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.380935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9256c-e580-4c9f-8287-6e0d26560c18 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.382446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 107/213] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:21.391601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.391601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.391601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.407767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:21.409203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:21.426048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9256c-e580-4c9f-8287-6e0d26560c18 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:21.485196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 107/214] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:21.547555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9256c-e580-4c9f-8287-6e0d26560c18 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:21.582754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:21.583663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.584121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.584565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.585915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:21.587536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:21.589363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 108/215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:21.595601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:21.621184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:21.654624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:21.661893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:21.714393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75996181-c8b0-4c4f-98b8-dea8f594b0d8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:21.751284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:21.751742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.752030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.752302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.753125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:21.754547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:21.756374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd95b54-43b4-4029-954d-356aa7a72806 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:21.779533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 108/216] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:21 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:45:21.780582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:21.797860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:21.798516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.798797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.799093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.812092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2c359f71-ee7d-4067-adc6-da52ee8e4e2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:21.817111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:21.818114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:21.819238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 109/217] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:21.826227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d691-643f-4ef6-a562-6af0f656305e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.826547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d691-643f-4ef6-a562-6af0f656305e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.826790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d691-643f-4ef6-a562-6af0f656305e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.828248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 110/218] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:21.830582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d691-643f-4ef6-a562-6af0f656305e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:21.919470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7063ca-691e-47e3-aa33-86b0f44b13ca None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:21.922862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:21.931197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:21.950477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d691-643f-4ef6-a562-6af0f656305e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:21.966475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:21.967590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.967590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.967778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.978948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-902013b8-448b-45ef-b5cf-94f3d1d1fd93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:21.984850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:21.985815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:21.986619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 109/219] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:21.987279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.987595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.987893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.993098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:21.993705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:21.994012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:21.994282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:21.994992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:21.995404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:21.996931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.006687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 111/220] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.007537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.008825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:22.010467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:22.109972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 110/221] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:22.111196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.130746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c9382-5807-42d1-94a3-49766ca31e87 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.174873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.175793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.176282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.177381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.178293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:22.178779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.180594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.192325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 112/222] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.197726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.201516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.238994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.242745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7566a9-f032-45f7-9938-6fbec084392b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.279522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.280528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.280989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.281438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] PATH_INFO: `/v3/users/c2dcd8c596234039b4fd7f21cf090f90` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.300009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd8c596234039b4fd7f21cf090f90)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.303054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.305073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 111/223] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/users/c2dcd8c596234039b4fd7f21cf090f90 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.310549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.329089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70db91-a514-426b-8f5e-1287e392048e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.373582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.375010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.375602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.376052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.377407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.379096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.380881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 113/224] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.391288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.437512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af661e80-66d6-482a-bed4-4c02af2d42cd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.472627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.473024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.473303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.473570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.496820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.510154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.535580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ef4d0-6d93-4cbd-917d-9e2d86904606 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.560095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.562797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.572080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.572707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.573504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.573799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.575223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.576788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.578687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 114/225] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.579096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.698365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47728-f305-4a9f-ab82-017df0654065 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.699549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 112/226] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:45:22.700636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.750264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.751645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.752138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.762791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.769716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:22.770289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:22.777850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:22.778759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:22.779531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:22.780145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 115/227] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:22.781263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.821348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef30416-45cf-4bf8-99af-210837cd2106 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.859005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.859847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.860192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.860573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/users/80629860440c468aa06d38511fc80e20/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.862756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, user_id=80629860440c468aa06d38511fc80e20, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.864322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.882052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 113/228] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:22 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/users/80629860440c468aa06d38511fc80e20/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:22.887112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:22.914936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ac4e-de2a-4e35-8b44-58b39be2a16b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:22.956544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:22.957189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:22.957471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:22.957761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:22.958936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:22.960510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:22.962449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 116/229] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:22.969535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:23.056612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:23.086773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd397-7772-4a7e-9f78-c1e8895dd0bb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:23.103161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348a4c7-a7bf-4d1b-b649-f574a16d1249 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:23.106900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:23.127899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:23.128524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:23.128804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:23.129070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:23.129762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:23.130089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:23.131470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:23.147197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 114/230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:23.147398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:23.148002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:23.148281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:23.148559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:23.149312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=c2dcd8c596234039b4fd7f21cf090f90) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:23.149625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:23.151212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:23.152914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:23.158290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 117/231] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:22 2022] GET /identity/v3/role_assignments?user.id=c2dcd8c596234039b4fd7f21cf090f90&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:23.325608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d3a31-746f-4f16-bffb-587f5d09b480 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:23.379480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:23.380056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:23.380337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:23.380615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:23.381801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:23.383274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:23.384875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 115/232] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:23 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:23.390795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:23.513778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33881d3b-c07b-4ca5-8cf8-e973c7a69f8c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:23.573760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:23.581144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:23.581856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:23.582141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:23.582416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:23.583657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:23.585185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e0789-8315-4b64-a159-b43879e89b8d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:23.588137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 118/233] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:23 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:23.594787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:23.692555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:23.769536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dbec8-3de4-4015-a1d4-efdf5d3470eb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:23.821934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:23.822610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:23.823413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:23.823690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/users/80629860440c468aa06d38511fc80e20/roles/b1177515548b419b92d3359f27435e02` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:23.825929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, user_id=80629860440c468aa06d38511fc80e20, role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:23.827384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00ef42-420d-4ade-a1b5-024c7b96b7f8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:23.848835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 116/234] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:23 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/users/80629860440c468aa06d38511fc80e20/roles/b1177515548b419b92d3359f27435e02 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:24.312138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633901-4325-4249-b69b-52a1abdd422c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:24.312533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633901-4325-4249-b69b-52a1abdd422c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:24.312711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633901-4325-4249-b69b-52a1abdd422c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:24.314395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 119/235] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:24.320670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a3ced-a90b-4e84-b09d-d346df64cdb2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:24.320954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a3ced-a90b-4e84-b09d-d346df64cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:24.321233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a3ced-a90b-4e84-b09d-d346df64cdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:24.349234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-db5a3ced-a90b-4e84-b09d-d346df64cdb2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:24.350483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-db5a3ced-a90b-4e84-b09d-d346df64cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:24.445668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 117/236] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:24.732175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a59f-c835-4746-ad9c-8fc16bef6ed1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:24.732651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a59f-c835-4746-ad9c-8fc16bef6ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:24.732651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139a59f-c835-4746-ad9c-8fc16bef6ed1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:24.734405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 120/237] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:24.739598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:24.739890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:24.740165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:24.762746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:24.764424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:24.804922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:24.855330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 118/238] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:24.906439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139a59f-c835-4746-ad9c-8fc16bef6ed1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.031616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139a59f-c835-4746-ad9c-8fc16bef6ed1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.086559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.087206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.087487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.087758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.108532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:25.113911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:25.114771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:25.115556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 121/239] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:25.121313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.154160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.247315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:25.288355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1590a-d86e-4e17-a7d9-55abdfc20bad None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.328229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c05bf-ee1c-46be-8a23-b87ed65f282e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.359791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.360446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.360674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.360945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.361623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:25.361951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:25.363341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:25.383416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 119/240] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:25.391355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.398154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.398816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.399091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.399357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.402583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:25.404413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:25.407398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 122/241] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:25.412549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.432057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:25.504967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627bcbe-2d2c-474a-8347-4b7d951d856b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.542180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.543135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.543559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.543973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] PATH_INFO: `/v3/users/0cfbb9891d484ca2b8495b5759ee70ce` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.558466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] RBAC: Authorizing `identity:get_user(user_id=0cfbb9891d484ca2b8495b5759ee70ce)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:25.560355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7c7fa-a4a5-45a8-a2a0-bb8d798d1a8c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:25.562490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 120/242] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:25 2022] GET /identity/v3/users/0cfbb9891d484ca2b8495b5759ee70ce => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:25.564098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ee526-244a-4881-86c4-9c3ad3644859 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.564527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ee526-244a-4881-86c4-9c3ad3644859 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.564944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ee526-244a-4881-86c4-9c3ad3644859 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.566758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 121/243] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:25.571702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.572394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.573365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.579470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903eacd-e89b-4d20-9532-c19dd5914692 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.597345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:25.598977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:25.607876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:25.617467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.618308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.618723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.619208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.620085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:25.621662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:25.645037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 123/244] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:25 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:45:25.653779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.684561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 122/245] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:25.845617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de9ab4-a5f2-4b58-b311-67f927c955aa None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:25.894457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:25.895094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:25.895373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:25.895646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:25.974310 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:25.978543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.003520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.005183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:26.006812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d1cb-b1d6-479d-82f7-ca638397135d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:26.118983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfbd03-fcb4-460f-a6e6-42864836b4d6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.147796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 124/246] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:26.149101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56928f3-650a-4a69-8dd9-28765363e84b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.149386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56928f3-650a-4a69-8dd9-28765363e84b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.149663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56928f3-650a-4a69-8dd9-28765363e84b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.151492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 125/247] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:26.158698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.159378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.159665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.171823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.172477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.172751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.173019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.183184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:26.184403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:26.188092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:26.193264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:26.194197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:26.194991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:26.195580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 123/248] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:26.201185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.266383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 126/249] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:26.267464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.283974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.296190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.331620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e60e6-1a5c-4103-ad60-59d6cfafa0d2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.377471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.378107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.378376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.378651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.380585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:26.382078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:26.386265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 124/250] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:26.392441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.417023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227da7-298c-461e-b84e-dc4add7e3162 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.498932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97297d46-93dd-46af-8022-d87114711067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.500012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97297d46-93dd-46af-8022-d87114711067 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.500680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97297d46-93dd-46af-8022-d87114711067 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.500966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97297d46-93dd-46af-8022-d87114711067 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.501656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97297d46-93dd-46af-8022-d87114711067 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:26.502004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97297d46-93dd-46af-8022-d87114711067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:26.504040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97297d46-93dd-46af-8022-d87114711067 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:26.524345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 127/251] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:26.528579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97297d46-93dd-46af-8022-d87114711067 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.556076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c963e-2c6b-4697-bec2-fc65dd632b5e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.623547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.624514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.625013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.625495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.626449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=0cfbb9891d484ca2b8495b5759ee70ce) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:26.626959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:26.628665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:26.643745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 125/252] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/role_assignments?user.id=0cfbb9891d484ca2b8495b5759ee70ce&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:26.645043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.657380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.676112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.691937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97297d46-93dd-46af-8022-d87114711067 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.735785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.735785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.735785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.735785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.736923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:26.738523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:26.742028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 128/253] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:26.748524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:26.907525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51228552-6029-4890-aecb-7bc118b17273 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.911180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cb9ab-b5af-49f1-8021-38b3250a7c8c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:26.962514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.963073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:26.963812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.963985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.964239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.964337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.964525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.964790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.976851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:26.977560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8a56523c-ff20-4676-a6a7-bc5496c718ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:26.983158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:26.984033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:26.985283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 126/254] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:26.986673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90ed50-773f-44c9-b24d-6712b6e2b1ee None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:26.986832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90ed50-773f-44c9-b24d-6712b6e2b1ee None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:26.988520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90ed50-773f-44c9-b24d-6712b6e2b1ee None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:26.988520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 127/255] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:26.993287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90ed50-773f-44c9-b24d-6712b6e2b1ee None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.050238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.051722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311da89-12d4-43c4-83ea-748eb9be95e6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.164596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:27.171417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 129/256] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:45:27.172910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.173204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.173486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.200808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:27.202162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:27.242890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:27.275705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90ed50-773f-44c9-b24d-6712b6e2b1ee None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:27.282863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 130/257] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:27.283496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.347814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:27.349020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.349592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.350077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.350951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:27.351472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.353245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.368840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 128/258] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:27.374621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.420923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d80b2-9457-42e7-be23-b65f8cdf4f99 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:27.458215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:27.458859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.459142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.459561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.460538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.462022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.463694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 131/259] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:27 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:27.472315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.530774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9c5bd-d693-4fc8-a22f-af2c931aebac None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:27.569088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:27.594189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:27.603462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:27.604943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.606268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.606695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] PATH_INFO: `/v3/users/c2dcd8c596234039b4fd7f21cf090f90` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.608780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd8c596234039b4fd7f21cf090f90)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.611450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.615415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 129/260] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:27 2022] GET /identity/v3/users/c2dcd8c596234039b4fd7f21cf090f90 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:27.620632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.710829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858e39-d352-4948-bde0-5b3a0a3ac4d4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:27.792637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:27.793614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.794048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.794490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.795349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=80629860440c468aa06d38511fc80e20) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:27.795835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.797414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.809368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 132/261] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:27 2022] GET /identity/v3/role_assignments?user.id=80629860440c468aa06d38511fc80e20&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:27.821341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcdeac-8971-436b-a637-99a23586d9f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:27.881249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:27.886701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:27.886989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:27.887266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:27.889786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:27.891304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:27.893393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 130/262] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:27 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:27.898996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:27.937094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:28.045412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:28.116315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff400d6-431a-425b-bada-878156207e44 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:28.132304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ef61b-f059-4a67-b1f2-8c801208d98a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:28.147554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:28.178742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:28.178742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.178742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.178742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.189622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:28.190839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.191641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.192118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/users/c2dcd8c596234039b4fd7f21cf090f90/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.194528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, user_id=c2dcd8c596234039b4fd7f21cf090f90, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:28.196208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:28.199207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:28.205546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:28.206505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:28.207826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 131/263] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:28.218951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:28.222727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 133/264] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:27 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/users/c2dcd8c596234039b4fd7f21cf090f90/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:28.337808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388276e7-9f01-435f-b0e5-b8c59177f1e7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:28.371190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:28.379670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:28.380496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.380952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.381431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.382183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:28.382485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:28.383930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:28.393723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 132/265] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:28.399711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:28.538877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c99fab-e371-4331-a22d-bfcaad462105 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:28.578088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:28.579074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.579554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.580023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] PATH_INFO: `/v3/users/80629860440c468aa06d38511fc80e20` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.581362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] RBAC: Authorizing `identity:get_user(user_id=80629860440c468aa06d38511fc80e20)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:28.583157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb869ea-d69d-4b76-8dfd-df24086d463b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:28.590046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 134/266] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:28 2022] GET /identity/v3/users/80629860440c468aa06d38511fc80e20 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:28.592009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:28.709782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99144f6-2f57-4e10-83bf-2e2d2718ad6e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:28.744231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:28.745359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.745637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.745934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.829976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b38a-26ed-489b-80f3-54c4f6639085 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.830602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b38a-26ed-489b-80f3-54c4f6639085 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.831117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b38a-26ed-489b-80f3-54c4f6639085 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.832943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 135/267] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:28.839425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:28.839954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:28.840481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:28.841623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:28.862274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:28.863766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:28.878676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:28.879873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:28.928055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:29.012031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 136/268] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:29.064788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 133/269] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:29.070375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.204943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f123-4dae-4b2d-a31b-c5fa45fd66ac None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:29.253193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:29.253871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.254150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.254419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.255534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:29.256993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c17175-8b6a-4d65-b72c-dbd5f06fd0e5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:29.262886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 137/270] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:29.270829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.313366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0103-5736-40fd-a7ab-f4cd9c8cf0ce None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.313673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0103-5736-40fd-a7ab-f4cd9c8cf0ce None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.313965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0103-5736-40fd-a7ab-f4cd9c8cf0ce None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.317758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 138/271] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:29.320531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.320802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.321070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.338962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:29.340198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:29.378835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:29.429798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 139/272] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:29.430748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.442316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:29.457678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0a16d-d136-4b0d-88c8-f90e9d17992d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:29.528443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:29.529056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.529331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.529602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.530362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=80629860440c468aa06d38511fc80e20) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:29.530677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:29.532123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:29.546409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 134/273] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/role_assignments?user.id=80629860440c468aa06d38511fc80e20&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=b1177515548b419b92d3359f27435e02 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:29.608371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54916db-c78b-45f0-8722-6da09a8d1f48 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:29.671751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:29.672415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.672695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.672966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.694998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:29.700012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:29.700901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:29.701688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 140/274] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:29.711005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.777378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.813261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:29.832054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f95bc4-b969-4159-a7a0-3aa89fdc1358 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:29.894201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:29.894827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.895100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.895371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.896045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:29.896371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:29.897849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847551-05e6-45bd-827f-bad56a495851 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:29.907878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 135/275] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:29.908911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54769-4925-4e3e-99f4-1d2a2d6c33fa None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.909194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54769-4925-4e3e-99f4-1d2a2d6c33fa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.909464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b54769-4925-4e3e-99f4-1d2a2d6c33fa None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.913781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 136/276] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:29.913781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54769-4925-4e3e-99f4-1d2a2d6c33fa None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:29.919516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974c378-d0b7-4149-aafb-a2d019c2f479 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:29.935549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:29.955029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:29.955657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.955935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.956202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.957298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:29.958768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48ab0c-c95e-42dd-8a97-7e9e354b236f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:29.960469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 141/277] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:29.961433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:29.961659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:29.961947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:29.981102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:29.982331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:30.028875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b54769-4925-4e3e-99f4-1d2a2d6c33fa None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.051628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 142/278] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:30.052606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.073123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.073782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.074067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.074341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] PATH_INFO: `/v3/users/1023e1472e514fac8c6d91373634b6af` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.080681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:30.091925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] RBAC: Authorizing `identity:get_user(user_id=1023e1472e514fac8c6d91373634b6af)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:30.093554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:30.095675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 137/279] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:29 2022] GET /identity/v3/users/1023e1472e514fac8c6d91373634b6af => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:30.101227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.180707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd0dc-c7d6-4664-adb7-9635859a5b74 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.218320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:30.226380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.226857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c32a4-1f26-4de9-83fc-025a7feed13e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.227696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.228201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.229327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.231072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:30.232746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:30.262936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 143/280] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:30 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:30.271734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.272772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.273248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.273798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.274784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, user.id=1023e1472e514fac8c6d91373634b6af, scope.system=all) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:30.275331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:30.277798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:30.284606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 138/281] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 07:45:30 2022] GET /identity/v3/role_assignments?user.id=1023e1472e514fac8c6d91373634b6af&scope.system=all&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:30.331705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.429996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:30.464605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f43a2-8ed4-4e66-af1b-5ce5b8798001 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.506266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.506841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.507117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.507385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.519127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:30.524871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:30.526358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:30.527165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 144/282] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:30.533712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.647368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2c40-e913-4cf5-b314-6f4abaf4f0b1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.685199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.685848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.686379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.686656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.687334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:30.687638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:30.689029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed4fc3-0bfc-44c9-90e0-64a36ea9c2bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:30.695449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:30.700475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 139/283] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:30.706581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.826245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676b10b-47fc-46ba-909f-840ad58a093a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:30.849130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8d3f2-2304-413f-a889-79c515d8cfe7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.849673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8d3f2-2304-413f-a889-79c515d8cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.849673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8d3f2-2304-413f-a889-79c515d8cfe7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.854925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 140/284] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:30.856790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.857076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.857348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.868921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:30.869952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:30.870461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:30.870949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] PATH_INFO: `/v3/users/0cfbb9891d484ca2b8495b5759ee70ce` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:30.872285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=0cfbb9891d484ca2b8495b5759ee70ce)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:30.873983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:30.875777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 145/285] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:30 2022] GET /identity/v3/users/0cfbb9891d484ca2b8495b5759ee70ce => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:30.882639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:30.886206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:30.905756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:30.905756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:30.995004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66d92-9353-4728-88d7-1c8497911ec5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:31.031447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:31.032139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:31.032424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:31.032698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:31.033892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:31.041756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:31.041756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 146/286] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:30 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:31.043571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:31.096587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 141/287] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:31.130166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:31.177575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c605e73-9832-4dc7-b293-7af47b3989c2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:31.212834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:31.213527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:31.213841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:31.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/users/0cfbb9891d484ca2b8495b5759ee70ce/roles/17eb64f9d07f42c38e67fd8bbc093a79` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:31.216040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, user_id=0cfbb9891d484ca2b8495b5759ee70ce, role_id=17eb64f9d07f42c38e67fd8bbc093a79)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:31.217535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:31.246414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 147/288] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:45:31 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/users/0cfbb9891d484ca2b8495b5759ee70ce/roles/17eb64f9d07f42c38e67fd8bbc093a79 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:31.463715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:31.588953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b599d-bcbe-4d21-9160-59d62bd90afe None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:31.620170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:31.628353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:31.628990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:31.629267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:31.629535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:31.641004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:31.645881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:31.646744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:31.647617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 142/289] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:31.654369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:31.773279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b6b7d-9e72-47b2-bb6a-d5ae698341f6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:31.802578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:31.808513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:31.809162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:31.809604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:31.810066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:31.811357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:31.811803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:31.813606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:31.823171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 148/290] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:31.828709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:31.981843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667cf6-3f73-474a-ae26-7aabcbb448e3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:32.026352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:32.027262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:32.027696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:32.028151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] PATH_INFO: `/v3/users/c2dcd8c596234039b4fd7f21cf090f90` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:32.034414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd8c596234039b4fd7f21cf090f90)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:32.036015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:32.037785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 143/291] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:31 2022] GET /identity/v3/users/c2dcd8c596234039b4fd7f21cf090f90 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:32.044157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:32.156651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb311e7-0996-4e6e-b03e-823fdf4d90de None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:32.192419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:32.193260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:32.193684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:32.194149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:32.255427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:32.267196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:32.268847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:32.370540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 149/292] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:32.379046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:32.457929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:32.512089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0668d3-fa39-4319-ab0f-ffc7b62e162e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:32.555053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:32.555927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:32.556404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:32.557342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:32.562435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:32.564066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a32014-47ad-41dc-99e5-7e31ba89f1a1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:32.565899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 144/293] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:32 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:32.571709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:32.671261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:32.679914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258192f-23f3-44ea-8982-9fa2db84c3fe None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:32.716287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:32.717131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:32.717551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:32.718061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:32.718929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=c2dcd8c596234039b4fd7f21cf090f90) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:32.719377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:32.720959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e606f-8ddc-4988-9939-36bb8347a48d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:32.727610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 150/294] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:32 2022] GET /identity/v3/role_assignments?user.id=c2dcd8c596234039b4fd7f21cf090f90&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:33.054047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4205-6550-4313-86e2-a02f20b289f1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.055053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4205-6550-4313-86e2-a02f20b289f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.055541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf4205-6550-4313-86e2-a02f20b289f1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.057685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 145/295] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:33.063585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d5773-0f85-47fc-9310-530ab71333fc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.064143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d5773-0f85-47fc-9310-530ab71333fc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.064558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d5773-0f85-47fc-9310-530ab71333fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.083173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1c2d5773-0f85-47fc-9310-530ab71333fc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:33.084558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1c2d5773-0f85-47fc-9310-530ab71333fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:33.131162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bf04-ad4b-4b02-b9fa-c0d6ad786d39 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.131162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bf04-ad4b-4b02-b9fa-c0d6ad786d39 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.131458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56bf04-ad4b-4b02-b9fa-c0d6ad786d39 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.132717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 146/296] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:33.140036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999404-02d5-4545-9cf6-df1e1b3209aa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.140183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999404-02d5-4545-9cf6-df1e1b3209aa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.140384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999404-02d5-4545-9cf6-df1e1b3209aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.161098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 151/297] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:33.162426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ca999404-02d5-4545-9cf6-df1e1b3209aa None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:33.168695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca999404-02d5-4545-9cf6-df1e1b3209aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:33.239815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 147/298] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:33.359742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcd82-18f4-4156-96e3-49c410336e92 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.360547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcd82-18f4-4156-96e3-49c410336e92 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.360964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fcd82-18f4-4156-96e3-49c410336e92 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.362717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 152/299] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:33.368364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.368724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.369033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.390720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:33.391915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:33.444460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:33.444994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fcd82-18f4-4156-96e3-49c410336e92 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.464408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 148/300] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:33.527530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.535297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:33.557511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fcd82-18f4-4156-96e3-49c410336e92 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:33.604546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:33.605202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.605481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.605771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.621547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:33.627994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:33.628882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:33.629671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:33.630310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 153/301] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:33.638671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.640024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c46f8-7f95-494f-8c02-b90fdcdc9c29 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:33.682861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:33.684065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.684513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.684929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.699486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:33.707153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:33.708238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:33.709075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:33.716743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 149/302] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:33.718559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.758451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd99c-e513-4d19-9fac-1b371c8db57f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:33.774357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:33.797391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:33.798034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.798312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.798585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.799247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:33.799556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:33.800938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:33.809417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 154/303] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:33.810243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.823328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:33.836651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b1a81-d678-4b83-b79b-645e0790db01 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:33.872677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:33.873318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.873596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.873910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.874595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:33.874892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:33.876278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:33.888874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 150/304] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:33.889688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:33.929892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ccaa-c3dc-48b5-a215-53fd6b50b76a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:33.969704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:33.970357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:33.970785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:33.971064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:33.972950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:33.974445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:33.976219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 155/305] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:33.978611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.012387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffed4c-bfc8-4c52-ba60-4199e12498a9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.021623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.048193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.048834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.049115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.049390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] PATH_INFO: `/v3/users/c72f07247ba546379106676b4cd4b418` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.055466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.063728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] RBAC: Authorizing `identity:get_user(user_id=c72f07247ba546379106676b4cd4b418)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:34.065187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:34.066849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 151/306] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/users/c72f07247ba546379106676b4cd4b418 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:34.067891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.084252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a49b8-677b-483f-a1aa-57e65b7582a5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.116290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.116923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.117203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.117477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] PATH_INFO: `/v3/users/1023e1472e514fac8c6d91373634b6af` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.118637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] RBAC: Authorizing `identity:get_user(user_id=1023e1472e514fac8c6d91373634b6af)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:34.120639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34d6c3-97ec-4e4b-a435-3f9a8e133448 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:34.122407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 156/307] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:33 2022] GET /identity/v3/users/1023e1472e514fac8c6d91373634b6af => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:34.123401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5912e-3599-42ce-bdfb-84d4191904e3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.123630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5912e-3599-42ce-bdfb-84d4191904e3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.123904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5912e-3599-42ce-bdfb-84d4191904e3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.125101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 157/308] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:34.126482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5912e-3599-42ce-bdfb-84d4191904e3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.171127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff324b-097d-4836-9a70-c746e5079a94 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.205299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.205942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.206219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.206490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] PATH_INFO: `/v3/groups` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.207124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:34.208548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:34.227186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5912e-3599-42ce-bdfb-84d4191904e3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.236624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 152/309] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Sep 13 07:45:34 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:34.237519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.268961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.269587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.269897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.270172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.270914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, user.id=c72f07247ba546379106676b4cd4b418, scope.system=all) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:34.271223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:34.272702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04a3bf-8229-4618-a410-f5c99e3fc8f5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:34.278342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 158/310] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 07:45:34 2022] GET /identity/v3/role_assignments?user.id=c72f07247ba546379106676b4cd4b418&scope.system=all&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:34.279695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.279985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.280257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.305599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:34.307546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:34.311524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.328011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.396946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb2059-dcf1-4300-a5b7-b556ba3d4fcd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.408297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 159/311] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:34.440231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.440966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.441252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.441528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] PATH_INFO: `/v3/system/users/1023e1472e514fac8c6d91373634b6af/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.443212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1023e1472e514fac8c6d91373634b6af, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:34.444689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:34.458399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 153/312] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Sep 13 07:45:34 2022] PUT /identity/v3/system/users/1023e1472e514fac8c6d91373634b6af/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:34.685841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.785441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42617b5-375a-4589-acf8-35160da45e91 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.824110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.824747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.825022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.825289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:34.832485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.836162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:34.843274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:34.844626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:34.846521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 160/313] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:34.850411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:34.948623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:34.960076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cfd24-f3fd-41d6-9628-e469f3af1e1c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:34.998290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:34.998910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:34.999191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:34.999465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:35.000139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:35.000444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:35.001848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:35.010952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 154/314] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:35.016356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:35.124028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f912e-c868-41ca-aa33-e66bea9fdec4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:35.159110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:35.159741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:35.160016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:35.160371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] PATH_INFO: `/v3/users/0cfbb9891d484ca2b8495b5759ee70ce` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:35.161562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=0cfbb9891d484ca2b8495b5759ee70ce)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:35.163046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:35.164923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 161/315] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:35 2022] GET /identity/v3/users/0cfbb9891d484ca2b8495b5759ee70ce => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:35.170480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:35.285161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e467a-ef83-40fd-a3ff-733c8047ea0b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:35.319189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:35.319796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:35.320076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:35.320348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:35.346042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:35.389458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:35.390977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:35.492455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 155/316] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:35.498373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:35.578105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:35.611746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df5470-cbde-490e-a439-d7e8799b4cf1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:35.646129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:35.646762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:35.647037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:35.647301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:35.648419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:35.649947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e14b16-7f05-4478-b039-3de7c45bae02 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:35.651889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 162/317] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:35 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:35.657720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:35.718172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:35.767914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35cb373-63f7-451c-9364-0fe38c3d3c71 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:35.802924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:35.803536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:35.803815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:35.804085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:35.804818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, scope.project.id=e08237d356514f16a7e8163f4b657717, user.id=0cfbb9891d484ca2b8495b5759ee70ce) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:35.805128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:35.807169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9fa4b-a93a-4598-b501-cbd73b66efd5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:35.813868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 156/318] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:45:35 2022] GET /identity/v3/role_assignments?user.id=0cfbb9891d484ca2b8495b5759ee70ce&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:36.558762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eeafd-e66c-4dc6-8019-0aad87d957e6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.559282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eeafd-e66c-4dc6-8019-0aad87d957e6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.559356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eeafd-e66c-4dc6-8019-0aad87d957e6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.560191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ae6db-c56d-416b-bab3-95a83ff72557 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.560552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ae6db-c56d-416b-bab3-95a83ff72557 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.560864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ae6db-c56d-416b-bab3-95a83ff72557 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.560973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 163/319] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:36.562411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 157/320] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:36.566299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b843e00-7181-4eff-ac08-d04f08bbabe1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.566586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b843e00-7181-4eff-ac08-d04f08bbabe1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.566862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b843e00-7181-4eff-ac08-d04f08bbabe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.567127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a2d1-760e-45f5-9e45-4fe30af7c2ac None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.567409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a2d1-760e-45f5-9e45-4fe30af7c2ac None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.567682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a2d1-760e-45f5-9e45-4fe30af7c2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.585300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0b843e00-7181-4eff-ac08-d04f08bbabe1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:36.586517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0b843e00-7181-4eff-ac08-d04f08bbabe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:36.588838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2a01a2d1-760e-45f5-9e45-4fe30af7c2ac None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:36.590050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2a01a2d1-760e-45f5-9e45-4fe30af7c2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:36.661111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 164/321] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:36.669780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 158/322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:36.722785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a5053-b7ec-44cc-a53d-f9baa0bd6879 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.723071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a5053-b7ec-44cc-a53d-f9baa0bd6879 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.723345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a5053-b7ec-44cc-a53d-f9baa0bd6879 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.724821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 165/323] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:36.729930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:36.730290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:36.730628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:36.748382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:36.749625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:36.792273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:36.818126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 159/324] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:36.938100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a5053-b7ec-44cc-a53d-f9baa0bd6879 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:36.940074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.003955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.044504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddd384-6ae3-489a-9340-e72b08dc3605 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.044784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a5053-b7ec-44cc-a53d-f9baa0bd6879 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.080433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.080715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.080999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.081251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.081345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.081527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.081619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.081836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.082930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.084654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.086252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 160/325] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:37.087158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.096269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:37.101047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:37.102083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:37.102871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 166/326] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:37.103558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.133435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.187397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c23a6-7db7-4926-a86e-88a94c2719de None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.202922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f5671-751e-49f3-aa7e-e7619c15e911 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.220007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.220628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.220899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.221191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.225717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.231061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:37.235942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:37.236823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:37.237931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:37.238751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.238850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 161/327] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:37.238927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.238927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.238927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] PATH_INFO: `/v3/groups` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.239095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.239168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.240133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.257644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 167/328] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Sep 13 07:45:37 2022] POST /identity/v3/groups => generated 226 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:37.258836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.364020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8c17-bc8a-489a-b219-141c86159cdd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.366744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.378748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5428a32-c1e1-4309-9909-b72c857aab9d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.404158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.404835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.405116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.405389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.406118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:37.406418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.407820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.415283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.416783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.417209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.417622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.422473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 162/329] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:37.422617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:37.422617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.423893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.423996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.434088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 168/330] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:37.439857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.494582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.544394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cf7d9-b3c5-4fe4-9ddc-bf5628df6124 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.565100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec43c32-f784-42e3-b689-608c3f4702b7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.588728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.590115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.590392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.590670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] PATH_INFO: `/v3/users/c72f07247ba546379106676b4cd4b418` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.591859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] RBAC: Authorizing `identity:get_user(user_id=c72f07247ba546379106676b4cd4b418)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.593908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.595248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 163/331] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/users/c72f07247ba546379106676b4cd4b418 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:37.602813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.606620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.607432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.607858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.608127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] PATH_INFO: `/v3/users/1023e1472e514fac8c6d91373634b6af` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.609288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] RBAC: Authorizing `identity:get_user(user_id=1023e1472e514fac8c6d91373634b6af)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.610866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.612449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 169/332] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/users/1023e1472e514fac8c6d91373634b6af => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:37.618755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:37.645218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.729553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937361d-2476-4dc9-9200-4184a39de294 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.747284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937504-1fc9-433e-a71d-d51c9facf8cf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:37.762766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:37.777623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.779366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.779841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.780644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] PATH_INFO: `/v3/system/users/c72f07247ba546379106676b4cd4b418/roles/17eb64f9d07f42c38e67fd8bbc093a79` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.783021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c72f07247ba546379106676b4cd4b418, role_id=17eb64f9d07f42c38e67fd8bbc093a79)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.785501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc2e0f-2046-4a52-a011-351d029707ba None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.789251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:37.789957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:37.790179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:37.790452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:37.792203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] RBAC: Adding query filter params (role.id=6e7afd6453c542aaa20fbb52594a1522, user.id=1023e1472e514fac8c6d91373634b6af, scope.system=all) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:37.792203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:37.793571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ac355-eb20-4df5-9a68-2c30b84aa79c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:37.796451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 164/333] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Sep 13 07:45:37 2022] PUT /identity/v3/system/users/c72f07247ba546379106676b4cd4b418/roles/17eb64f9d07f42c38e67fd8bbc093a79 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:37.819009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 170/334] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 07:45:37 2022] GET /identity/v3/role_assignments?user.id=1023e1472e514fac8c6d91373634b6af&scope.system=all&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:39.770341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bd56-445b-4207-9792-4241a4c2a8c7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:39.771385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bd56-445b-4207-9792-4241a4c2a8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:39.771872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90bd56-445b-4207-9792-4241a4c2a8c7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:39.773432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 165/335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:39.779207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:39.780061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:39.780574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:39.801535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:39.803111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:39.855503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919156bd-aed8-49e8-8b53-e8c25c9a250a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:39.856127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919156bd-aed8-49e8-8b53-e8c25c9a250a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:39.856604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919156bd-aed8-49e8-8b53-e8c25c9a250a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:39.858396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 166/336] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:39.867615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22439d-d563-4237-8e5d-f37c3f1808b2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:39.867897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22439d-d563-4237-8e5d-f37c3f1808b2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:39.868170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22439d-d563-4237-8e5d-f37c3f1808b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:39.898189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7f22439d-d563-4237-8e5d-f37c3f1808b2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:39.899387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7f22439d-d563-4237-8e5d-f37c3f1808b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:39.915419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 171/337] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:40.000370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 167/338] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:40.239033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:40.361227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f136-b3cd-4bf2-9b41-9ff4d4b1773e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:40.362772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:40.378791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f830de-0d28-4890-9d12-b4ae8d5b88bd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.379395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f830de-0d28-4890-9d12-b4ae8d5b88bd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.379861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f830de-0d28-4890-9d12-b4ae8d5b88bd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.381666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 168/339] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:40.390989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.391551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.392019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.405066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:40.405714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.406020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.406294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.425432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:40.427021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:40.428479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:40.433484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:40.434409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:40.435783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 172/340] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:40.436719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:40.458171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:40.563680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 169/341] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:40.564613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:40.565402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63d6a-ebf2-444b-9037-438f72931f93 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:40.603093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:40.604088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.604587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.605052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.617327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:40.622703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:40.623607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:40.624409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:40.627067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 173/342] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:40.640086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:40.690165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:40.698798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b904818-3683-4164-b9a0-340ab1983ce9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:40.736819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:40.737789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.738255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.738706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.739574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:40.740054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:40.741622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:40.752101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 170/343] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:40.758332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:40.798140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:40.844349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4302f8d-c2c0-4c4e-9833-a2b3a2a5189b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:40.901087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bde067-4a42-4c8f-b76a-836cd6de5bbb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:40.939953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:40.940809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.941240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.941801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.950442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:40.951274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:40.951716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:40.952124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:40.952934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:40.953409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:40.955026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:40.965670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 174/344] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:40.971164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.019259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.021120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.046089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.133363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 171/345] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:41.134614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.173574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.204250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b0d78-8553-48c9-890a-9fe1df2f4bb8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.248214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596fb89-d4a7-440e-a3af-afa3fd93bcfc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.254703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.255336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.255616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.255887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.272003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.288487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.289143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.289429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.294181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.310222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:41.323692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:41.324678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:41.325481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 172/346] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:41.326187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.340507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.344220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.445669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.464482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 175/347] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:45:41.465870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.498142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0e4f1-87bf-46d0-afd1-b6b50a5c404f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.542092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.543259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.543743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.544219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.545589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.547276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.549110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 173/348] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:41.550170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.576847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09879a-2c0d-4b5d-acb5-5728adbc7252 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.614108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.617859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.619177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.619929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.620422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.621787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:41.622420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.624278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f27e02-1185-4505-a78a-003c1f83aaa5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.636144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 176/349] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:41.638337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21006-edc2-46d2-94b9-0b2232040907 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.638833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21006-edc2-46d2-94b9-0b2232040907 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.639302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21006-edc2-46d2-94b9-0b2232040907 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.641515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 177/350] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:41.642657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21006-edc2-46d2-94b9-0b2232040907 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.685072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39ed09-1826-4720-b3f4-c56ae42d4d8a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.728718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.740359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.742228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.742721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.743445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.745244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.747623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.750665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 174/351] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:41.751975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.801808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21006-edc2-46d2-94b9-0b2232040907 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.845069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.846681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.847195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.847664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.855144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.857247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da5596-89a9-4164-9ad1-878929795f57 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.859419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 178/352] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:41.861021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.861527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.862082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.868487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30c6d6-0655-472a-8e8b-869f6aa50751 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:41.883412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:41.885590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:41.891273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:41.914534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:41.915018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:41.915295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:41.915600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] PATH_INFO: `/v3/users/c72f07247ba546379106676b4cd4b418` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:41.916760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c72f07247ba546379106676b4cd4b418)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:41.918510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:41.920281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 175/353] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/users/c72f07247ba546379106676b4cd4b418 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:41.921321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:41.973420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 179/354] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:41.974632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.021712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:42.048071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20e45d-96a6-425f-8cae-4750457834b4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.090084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.091081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.091556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.092043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.093383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:42.094924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc58128-c952-4219-814f-a398c9b84967 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.095386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:42.097161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 176/355] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:42.099252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.135237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.135619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.135916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.136154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.136878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:42.137177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:42.138628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5016da-1de4-45bc-986f-1bab765aa326 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:42.145306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 180/356] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:41 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:42.146926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291fd7d-0157-4eff-86e9-322c8fb561d9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.147475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291fd7d-0157-4eff-86e9-322c8fb561d9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.147756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291fd7d-0157-4eff-86e9-322c8fb561d9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.149509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 181/357] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:42.154613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291fd7d-0157-4eff-86e9-322c8fb561d9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.182804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:42.278381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291fd7d-0157-4eff-86e9-322c8fb561d9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.281163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:42.305880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c60a6-0e7e-4ab8-b1e4-47505f6f6dd7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.320436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.321058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.321333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.321601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.322953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=b1177515548b419b92d3359f27435e02, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:42.323258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:42.324881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b211d74-3c9f-4289-9434-52eae9693402 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:42.331150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 182/358] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=b1177515548b419b92d3359f27435e02 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:42.332756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.333020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.333296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.364018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:42.367068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:42.372375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.373382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.374166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.374648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.375612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] RBAC: Adding query filter params (role.id=17eb64f9d07f42c38e67fd8bbc093a79, user.id=c72f07247ba546379106676b4cd4b418, scope.system=all) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:42.376218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:42.378162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:42.396491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 177/359] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/role_assignments?user.id=c72f07247ba546379106676b4cd4b418&scope.system=all&role.id=17eb64f9d07f42c38e67fd8bbc093a79 => generated 510 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:42.397776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.462136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 183/360] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:42.543675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054d08-d34d-453e-ac71-ca95c1e1bb8e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.559217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:42.591955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.592622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.592910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.593172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.613045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:42.619327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:42.620346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:42.621117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 178/361] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:42.627211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.757524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:42.763573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee440d-09dd-444a-91bf-ba1141077c73 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:42.808470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:42.809415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:42.809938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:42.810400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:42.811537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:42.812031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:42.813591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:42.826737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 184/362] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:42.832327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:42.871987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.004617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477a5db-088b-41e3-9abc-d9ef5c60ab92 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.046354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.047402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.048719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.049218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.085712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ac08d-6b97-476e-ad93-d109e3d956e8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.098431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.138212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.139185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.139289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:43.139749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.140237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.140721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54124195-eecd-4332-8d8f-a57b196a3acd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:43.154335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c14224a9-4d7c-4341-aeda-6965545dc779 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:43.161373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:43.162350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:43.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:43.167128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 185/363] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:43.168754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e526b0-4bfc-4577-bcdf-8b43f58c921a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.169341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e526b0-4bfc-4577-bcdf-8b43f58c921a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.169835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e526b0-4bfc-4577-bcdf-8b43f58c921a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.171664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 186/364] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:43.172831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e526b0-4bfc-4577-bcdf-8b43f58c921a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.200216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.268365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 179/365] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:43.274634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.276081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.276511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.301601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:43.303360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:43.306081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e526b0-4bfc-4577-bcdf-8b43f58c921a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.349255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.349907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.350188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.350461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.351152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:43.351456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:43.352878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:43.363574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 187/366] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:43.364707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.391646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.396378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 180/367] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:43.401942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.503181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.530507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfa3ea-ddf2-4a0f-bd39-5a95c47caefc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.584094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.584735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.585014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.585284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.588070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:43.589517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:43.592335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 188/368] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:43.595188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9051a-5e7a-4215-918b-a60f88edf815 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.598546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.658872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.659411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.659690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.659960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.771699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.775995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:43.777508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:43.779132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954036b-bbb3-4b7f-800e-bd0ece04fd08 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:43.827882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:43.827882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:43.828183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:43.828252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:43.829367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:43.831662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:43.834235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 189/369] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:43.835005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.855103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:43.915818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 181/370] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:43.915818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:43.972883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245ec28-2680-4e63-9f1b-d03eb23e4e92 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.024876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.026152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.027046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.027639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.031183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.036318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcce7e-80d8-494a-a590-ca5483374908 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.043644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:44.049284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:44.050289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 190/371] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:44.051746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.086556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.087566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.088043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.088688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.090096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=e08237d356514f16a7e8163f4b657717) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:44.090844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.093108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.101387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 182/372] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:43 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.102621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.154966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.183200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf9654-915b-428d-8d1d-dec8cfc0948e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.222831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.223557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.223906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.224257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.225571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.227167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.229456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 191/373] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.254793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5131d2-669d-4fc1-bf0b-4f786f0840a1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.255669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.298155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.299142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.299608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.300081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.300967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:44.302172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.304024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.315937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 183/374] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.322855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.360004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.428532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3bc6a-16b2-4658-8780-a73f950af602 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.461882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d422d-e2c3-4d96-ae19-cb079f91fab7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.487264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea75c-9698-4051-b26e-24742ead985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.488167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea75c-9698-4051-b26e-24742ead985d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.488612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea75c-9698-4051-b26e-24742ead985d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.489139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befea75c-9698-4051-b26e-24742ead985d None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.494550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befea75c-9698-4051-b26e-24742ead985d None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.496226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befea75c-9698-4051-b26e-24742ead985d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.498142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 192/375] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.498281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.499051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.499575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.499937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.504535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea75c-9698-4051-b26e-24742ead985d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.535370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.582678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.584659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.616455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.674426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befea75c-9698-4051-b26e-24742ead985d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.714545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 184/376] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:45:44.719892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.726487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.727271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.727649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.728579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.729412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=b1177515548b419b92d3359f27435e02, scope.project.id=e08237d356514f16a7e8163f4b657717) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:44.729864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.733024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa8657-3dc2-4985-9c68-af137494f98e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.738524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 193/377] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=b1177515548b419b92d3359f27435e02 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.829684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d31e-e907-44c4-a519-a3e1ba6bc0a5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.830042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d31e-e907-44c4-a519-a3e1ba6bc0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.830428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d31e-e907-44c4-a519-a3e1ba6bc0a5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.831951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 194/378] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:44.840012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.840012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.840012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.857360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6b16-8af9-40e8-aa2f-335fc77a1a1b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:44.861634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:44.862999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:44.899122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:44.913876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:44.915135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:44.915765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:44.916282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] PATH_INFO: `/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:44.917662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] RBAC: Authorizing `identity:get_project(project_id=dab10ae977dc434f9d22b1fb83ca19e8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:44.919406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:44.921567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 185/379] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:44.930801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:44.966232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 195/380] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:45.084956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598af30-998c-4d42-9af7-eb6aba5ab259 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:45.125175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:45.126103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.126532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.126954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] PATH_INFO: `/v3/groups/304d464f2de44165b83617a194129eac` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.136297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] RBAC: Authorizing `identity:get_group(group_id=304d464f2de44165b83617a194129eac)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:45.137984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:45.140152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 186/381] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:44 2022] GET /identity/v3/groups/304d464f2de44165b83617a194129eac => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:45.145830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:45.159599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:45.297527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a504137-f4be-4387-9ed4-4b6e2d6a3a64 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:45.336416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:45.337299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.337713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.338145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.339029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] RBAC: Adding query filter params (group.id=304d464f2de44165b83617a194129eac, role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=dab10ae977dc434f9d22b1fb83ca19e8) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:45.339483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:45.341095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:45.347540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 196/382] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:45 2022] GET /identity/v3/role_assignments?group.id=304d464f2de44165b83617a194129eac&scope.project.id=dab10ae977dc434f9d22b1fb83ca19e8&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:45.406429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:45.503700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:45.558314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9461b-8931-433c-9cf6-1df8b8c476fd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:45.604720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:45.605364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.605640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.605993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.622845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:45.628316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:45.629246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:45.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 187/383] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:45.635670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:45.729618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:45.767597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48cc28-0424-4b3b-9b78-f2e314acfded None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:45.807623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:45.808231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.808555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.808770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.809432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:45.809755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:45.811404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c620a9-c0f6-4f81-8d58-51997e00436f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:45.821829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 197/384] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:45.827389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:45.888731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3efa7-70d1-41d8-a11b-b693a9d9ee66 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.889014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3efa7-70d1-41d8-a11b-b693a9d9ee66 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.889287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3efa7-70d1-41d8-a11b-b693a9d9ee66 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.890694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 198/385] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:45.897068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:45.897386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:45.897602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:45.919466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:45.920637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:45.991359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357644b2-46a2-4261-b622-9e2cb17fabdb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:46.021951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:46.067385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:46.067921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:46.068192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:46.068456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:46.069773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:46.071469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:46.078531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 199/386] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:46.078758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 188/387] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:45 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:46.084190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:46.170202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:46.210856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0557-d45f-4425-ad70-29fe98461e46 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:46.251887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:46.252530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:46.252807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:46.253077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:46.254263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:46.255735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:46.257669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 200/388] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:46 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:46.263021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:46.397066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e4be-0ad0-4e59-a1c9-774b9c338eb6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:46.435383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:46.436305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:46.436580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:46.436850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/groups/da18bfcf8620408cae22d9b282e2eed9/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:46.438770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, group_id=da18bfcf8620408cae22d9b282e2eed9, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:46.440224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:46.459155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 189/389] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 07:45:46 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/groups/da18bfcf8620408cae22d9b282e2eed9/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:46.474729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:46.616650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09382f84-74e5-45a3-8c53-75cee72a6ddc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:46.655618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:46.661434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:46.662106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:46.662384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:46.662663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:46.678768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:46.684553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:46.685584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:46.687352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 201/390] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:46.692345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:46.770146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:46.824388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8870830-351f-49c3-acb1-bc392e68e5a0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:46.869288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:46.869947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:46.870226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:46.870494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:46.871172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:46.871473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:46.872852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:46.885110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 190/391] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:46.890952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:47.018708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f6bfb-1333-4fbf-8c4f-76692733fa3a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:47.057233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:47.057920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.058199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.058519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.059707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:47.061411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:47.063122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 202/392] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:46 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:47.068792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:47.200264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051620-7e22-42a1-8a0d-fd64078fc4b2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:47.215798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:47.246778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:47.247422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.247706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.247980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.249139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:47.250597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095b284-49bf-4d50-91a9-fb99f49dd52b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:47.252544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 191/393] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:47 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:47.259941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:47.295997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293792e1-a069-48e4-9bcc-09dbee417eb1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.296955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293792e1-a069-48e4-9bcc-09dbee417eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.297447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293792e1-a069-48e4-9bcc-09dbee417eb1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.299281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 192/394] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:47.305177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2897b6-f253-4714-9155-2d6f4f7fbda1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.305872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2897b6-f253-4714-9155-2d6f4f7fbda1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.306372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2897b6-f253-4714-9155-2d6f4f7fbda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.334037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0f2897b6-f253-4714-9155-2d6f4f7fbda1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:47.336533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0f2897b6-f253-4714-9155-2d6f4f7fbda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:47.362953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:47.446435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e78edc-13fa-42d4-9e0a-ab68642db4a4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:47.478899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 193/395] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:47.495741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:47.496702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.497201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.497808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/groups/da18bfcf8620408cae22d9b282e2eed9/roles/b1177515548b419b92d3359f27435e02` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.500210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] RBAC: Authorizing `identity:create_grant(project_id=4f072ceca65e4101b7fb4ce52db5662f, group_id=da18bfcf8620408cae22d9b282e2eed9, role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:47.501900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b29b85-7747-45f9-baf6-71d9188b601c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:47.525861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 203/396] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 07:45:47 2022] PUT /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f/groups/da18bfcf8620408cae22d9b282e2eed9/roles/b1177515548b419b92d3359f27435e02 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:47.558731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841d89-9b33-4376-8873-6b4fdde609e8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.559318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841d89-9b33-4376-8873-6b4fdde609e8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.559811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841d89-9b33-4376-8873-6b4fdde609e8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.561620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 194/397] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:47.567245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.567847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.568391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:47.590610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:47.592161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:47.671626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 204/398] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:47.808510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841d89-9b33-4376-8873-6b4fdde609e8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:47.943404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841d89-9b33-4376-8873-6b4fdde609e8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:47.951249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:47.984996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:47.985631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:47.985939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:47.986208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.001860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:48.007020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:48.007890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:48.008695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 195/399] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:48.016430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.070772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.087317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:48.181920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383639e-2260-405e-925f-1a5b5a76ba04 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.222772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.223490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.223769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.224030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.224696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:48.224992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:48.226502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:48.236204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 205/400] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:48.241956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.256104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5e2a-f037-462d-81fc-0349d08a14eb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.316629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.317767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.318052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.318475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.331117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:48.336605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:48.339301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:48.341269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 196/401] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:48.349755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.380490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:48.468267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a30cc5-675a-43df-9cce-41b959719d24 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.483384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:48.514226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.514872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.515153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.515428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.517836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:48.519291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:48.522925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 206/402] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:48.530767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.535054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417895-efdd-478d-8058-ee5681bd6bcd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.619061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.619689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.619973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.620244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.620925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:48.621231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:48.622677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c12292-3cc3-479c-943c-7b66d2a1f563 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:48.635324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 197/403] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:48.636724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc838c-1eb2-44aa-8ae7-9073275e9d46 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.637072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc838c-1eb2-44aa-8ae7-9073275e9d46 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.637414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc838c-1eb2-44aa-8ae7-9073275e9d46 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.643765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 198/404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:48.644924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc838c-1eb2-44aa-8ae7-9073275e9d46 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.667489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd9365-d4c2-4e8d-8559-d28a8652031b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.705105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.705880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.706224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.706558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.707755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:48.709259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee88cd-7880-40b3-922e-2e5eb24dda10 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:48.711279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 207/405] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:48.713393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.713710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.714073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.735209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:48.736417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:48.755075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:48.771207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc838c-1eb2-44aa-8ae7-9073275e9d46 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.795777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:48.819842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.820858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.821902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.822394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:48.823821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:48.824789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 208/406] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:48.825604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.826176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:48.828026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 199/407] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:48.833367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:48.954080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34543b85-84fe-408d-8e99-d3a0b4c22b12 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:48.997280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:48.998371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:48.998852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:48.999324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/groups/da18bfcf8620408cae22d9b282e2eed9/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.001551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, group_id=da18bfcf8620408cae22d9b282e2eed9, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.003218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.046902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 209/408] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 07:45:48 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/groups/da18bfcf8620408cae22d9b282e2eed9/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:49.053497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9489d-0edd-4482-a449-7e94edc999ac None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.094976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.095620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.095894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.096158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.097399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.099033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.101147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 200/409] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:48 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:49.106708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.175934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:49.176887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.237727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:49.251429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd1c9-0bfa-4ed4-b8e6-495de7696388 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.294924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.296657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.297097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.297514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717/groups/da18bfcf8620408cae22d9b282e2eed9/roles/b1177515548b419b92d3359f27435e02` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.302289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e08237d356514f16a7e8163f4b657717, group_id=da18bfcf8620408cae22d9b282e2eed9, role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.304616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.325900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 210/410] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 07:45:49 2022] PUT /identity/v3/projects/e08237d356514f16a7e8163f4b657717/groups/da18bfcf8620408cae22d9b282e2eed9/roles/b1177515548b419b92d3359f27435e02 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:49.334010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd659a-7275-4a81-ba92-efc44368bdac None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.387308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.388327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.388744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.389154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.402281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-183bb45b-b927-41e1-a198-57ef695a7215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:49.409715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:49.410795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:49.411810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:49.414010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 201/411] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:49.420587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.480937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e53c-fe84-41ba-81e0-9bccea143a2e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.481592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e53c-fe84-41ba-81e0-9bccea143a2e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.482290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891e53c-fe84-41ba-81e0-9bccea143a2e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.484254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 202/412] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:49.490442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.491077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.491550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.511526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:49.513071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:49.579063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129ec63-148f-41b6-9417-91e0dd69d8e4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.580832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:49.623031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.623659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.623936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.624206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.624875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:49.625173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.627140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.638288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 211/413] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:49.638914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 203/414] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:49.644496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.746832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09306e-4fef-4a97-8bb9-f784ad2646e2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.781913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.782769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.783196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.783614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] PATH_INFO: `/v3/groups/304d464f2de44165b83617a194129eac` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.784908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] RBAC: Authorizing `identity:get_group(group_id=304d464f2de44165b83617a194129eac)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.786548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.788562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 212/415] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:49 2022] GET /identity/v3/groups/304d464f2de44165b83617a194129eac => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:49.794770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.811351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:49.914823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49140c0-a9b4-4883-b3fc-c76088a763eb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:49.959690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:49.960686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:49.961166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:49.961656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] PATH_INFO: `/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:49.963570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=dab10ae977dc434f9d22b1fb83ca19e8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:49.965248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:49.967440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 204/416] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:49 2022] GET /identity/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:49.972902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:49.982670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:50.020283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:50.102618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:50.124388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1c08-b81c-46ac-bad1-073b06bb6ee8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:50.163630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bdca5-fe56-43d4-8edf-a970ee784637 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:50.185050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:50.185719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.186284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.186558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.208848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:50.217519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:50.218447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:50.219233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 205/417] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:50.227698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:50.235500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:50.236175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.236454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.236725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] PATH_INFO: `/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8/groups/304d464f2de44165b83617a194129eac/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.238771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] RBAC: Authorizing `identity:create_grant(project_id=dab10ae977dc434f9d22b1fb83ca19e8, group_id=304d464f2de44165b83617a194129eac, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:50.240211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9caa-1eca-4dba-94d2-c28c89837dab None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:50.268248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 213/418] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 07:45:49 2022] PUT /identity/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8/groups/304d464f2de44165b83617a194129eac/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:45:50.288934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e235-f0f0-44a0-85cf-c918fd0c9bf9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.289270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e235-f0f0-44a0-85cf-c918fd0c9bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.289540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e235-f0f0-44a0-85cf-c918fd0c9bf9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.291491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 214/419] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:50.300602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.300953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.301139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.334530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:50.337458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:50.418133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c4864-769f-423b-be1d-cb51ab699845 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:50.487467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 215/420] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:50.496457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:50.497080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.497359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.497629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.498342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:50.498644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:50.500042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:50.510531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 206/421] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:50.516237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:50.538416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:50.627870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd371b-5d2d-4894-9ba5-c9ad5af10893 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:50.679436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:50.680387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:50.681386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:50.681885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:50.777246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:50.779087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:50.808419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:50.832242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:50.842627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:50.966625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 216/422] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:50.978007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.044177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aec32-cc15-49a0-9291-42b8411749f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.120735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803277d-db7b-400b-aa22-879a60de837f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.139194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:51.143110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.143784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.144059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.144333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.161972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.162758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.163265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.163859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.166365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:51.167828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:51.169023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:51.169849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 217/423] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:50 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:51.174428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:51.175289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:51.176615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 207/424] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:51.180979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.182319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.280325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:51.329423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29b7d6-2c2b-4970-8e47-3a4a8d2307e4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.367692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.368523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.368904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.369280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.371305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:51.373015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:51.375349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 218/425] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:51 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:51.383685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.398046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72393bd4-4b39-4e9c-9cb5-6428bc15a558 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.461472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.463308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.463761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.464178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.465011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:51.465458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:51.467673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:51.483202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 208/426] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:51.489020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:51.490381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.576214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccce500-30e1-4da4-b97a-d2b6b1d067f1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.640897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.642991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.643812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.644426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.645996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:51.646659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:51.651374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158b65d-b929-4d3b-96f2-ce475e9ebbfd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:51.661273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 219/427] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:51 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 611 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:51.670172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc6d22-6246-458f-87fd-8a7714d51c4f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:51.708030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:51.725967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:51.727095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.727581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.728122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.825129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:51.826994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:51.945228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f54a1-c055-46ba-bc3d-b563141835f0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.945674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f54a1-c055-46ba-bc3d-b563141835f0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.946191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f54a1-c055-46ba-bc3d-b563141835f0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.947204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 209/428] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:51.948286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 220/429] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:51.953619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:51.960082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168222c7-5e56-4091-a7b4-11b45e94574e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:51.960365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168222c7-5e56-4091-a7b4-11b45e94574e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:51.960644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168222c7-5e56-4091-a7b4-11b45e94574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:51.985037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-168222c7-5e56-4091-a7b4-11b45e94574e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:51.987274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-168222c7-5e56-4091-a7b4-11b45e94574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:51.994157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.077708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 221/430] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:52.087578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdeb7d-04bc-4108-b5db-8c8fb34a2828 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.127682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24d58-b33c-4d7b-8fe6-5d48ef44c3e5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.128080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24d58-b33c-4d7b-8fe6-5d48ef44c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.128317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.128666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24d58-b33c-4d7b-8fe6-5d48ef44c3e5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.128958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.129235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.129504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] PATH_INFO: `/v3/projects/4f072ceca65e4101b7fb4ce52db5662f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.130691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 222/431] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:45:52.131291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] RBAC: Authorizing `identity:get_project(project_id=4f072ceca65e4101b7fb4ce52db5662f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:52.132756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:52.135224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 210/432] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:51 2022] GET /identity/v3/projects/4f072ceca65e4101b7fb4ce52db5662f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:52.141127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.141661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.142129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.142408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.169275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:52.170810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:52.234637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.264270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 223/433] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:52.277594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3432a-8eca-4e67-b119-b5396f9ef615 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.281394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.315074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.315716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.315996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.316266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.317406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:52.318884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:52.320615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 211/434] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:52.326356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.388433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.451678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f4701-2e83-44a7-bbf6-81f54a846b3c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.494170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.494802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.495337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.495613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.496355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=b1177515548b419b92d3359f27435e02, scope.project.id=4f072ceca65e4101b7fb4ce52db5662f) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:52.496664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:52.498165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:52.504563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 224/435] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=4f072ceca65e4101b7fb4ce52db5662f&role.id=b1177515548b419b92d3359f27435e02 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:52.516068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39acf3-ebb0-4f05-97ce-f5d2f1d6a64b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.558615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.559992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.561134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.561622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.562189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.575046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:52.580954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:52.581974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 07:45:52.582855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:52.589825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 212/436] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:52.592219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.613704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.625947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.681557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfea8-3b16-499b-9ac8-741a006a26f0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.722195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd371f25-fead-4a50-96a8-6bf74ab75d1f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.723222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.723868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.724149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.724423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.744718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:52.752146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:52.753123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:52.753979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 225/437] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:52.762076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.773298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.775192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.775192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.775192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.775762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:52.776064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:52.777460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:52.793073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 213/438] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:52.799693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.900557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183943d-6b50-4d0c-bb95-ad98456b6c15 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.902234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.915439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:52.935578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69f05d-c6cb-4d50-aa1f-ff3019efa2b0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:52.945295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.945985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.946318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.946593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.947279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:52.947828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:52.948947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31a95-7a76-42ed-82f0-688408c32112 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:52.958210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 226/439] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:52.959433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cf175-75a0-4e8e-abd6-b9636acde4c5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.959711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cf175-75a0-4e8e-abd6-b9636acde4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.959988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cf175-75a0-4e8e-abd6-b9636acde4c5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:52.961339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 227/440] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:52.965393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cf175-75a0-4e8e-abd6-b9636acde4c5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:52.977232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:52.978745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:52.979931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:52.980422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.051824 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.053276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c7d8-9e30-423a-9858-2912aa249b47 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.077872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cf175-75a0-4e8e-abd6-b9636acde4c5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.111081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.111690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.111967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.112237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.137208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.142088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.154381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 214/441] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 07:45:53.159205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.159205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.159205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.173145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:53.174355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:53.183833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.185277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.238861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 215/442] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:53.239601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.286630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 228/443] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:53.292001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.341928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961f77f-8127-40c7-8670-9071ba7ace3e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.355993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.360630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.377230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.377893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.378174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.378447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.379576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.381015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.382672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 216/444] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:53.388579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.411279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfdfc9-cb53-487d-b922-380dd26d2bb8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.449436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.450089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.450372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.450648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] PATH_INFO: `/v3/projects/e08237d356514f16a7e8163f4b657717` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.451776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] RBAC: Authorizing `identity:get_project(project_id=e08237d356514f16a7e8163f4b657717)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.453230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.454880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 229/445] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/projects/e08237d356514f16a7e8163f4b657717 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:53.460154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.504628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7323b-093a-47b6-b632-c2b03d888b3a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.540169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.540819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.541042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.541304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.542439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.543872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.545594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 217/446] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:53.546267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.575342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2806f9-5574-48c6-b480-d90c878e6846 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.594845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.604354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.609596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.610261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.610693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.610965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] PATH_INFO: `/v3/groups/da18bfcf8620408cae22d9b282e2eed9` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.612552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] RBAC: Authorizing `identity:get_group(group_id=da18bfcf8620408cae22d9b282e2eed9)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.614116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.615588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 230/447] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/groups/da18bfcf8620408cae22d9b282e2eed9 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:53.616340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.646872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b0bf5-6196-4e98-af3c-26684546fcca None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.679287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.679928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.680189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.680960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.691187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:53.695969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:53.697494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 218/448] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:53.698353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.719416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f300c3-503c-4c26-a55d-0e4645ddd750 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.753336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.753973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.754252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.754523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.755300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=6e7afd6453c542aaa20fbb52594a1522, scope.project.id=e08237d356514f16a7e8163f4b657717) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:53.755546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.756983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.763131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 231/449] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=6e7afd6453c542aaa20fbb52594a1522 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:53.763848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:53.808221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212f180-355a-4690-87b4-5e1ba0c10c90 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.821483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.847235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:53.851276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.853000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.853297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.853574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.854368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] RBAC: Adding query filter params (group.id=da18bfcf8620408cae22d9b282e2eed9, role.id=b1177515548b419b92d3359f27435e02, scope.project.id=e08237d356514f16a7e8163f4b657717) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:53.854682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.856184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.862835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 219/450] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/role_assignments?group.id=da18bfcf8620408cae22d9b282e2eed9&scope.project.id=e08237d356514f16a7e8163f4b657717&role.id=b1177515548b419b92d3359f27435e02 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:53.906227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3be26c-51be-41f1-9100-4b8ad7bf7bc7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:53.947275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:53.947906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:53.948190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:53.948465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:53.949152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:53.949459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:53.950910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:53.960584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 232/451] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:53.966656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:54.081566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e418b-54e7-4e31-8d57-863f1b70e0f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:54.116332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:54.116941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:54.117221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:54.117495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:54.161018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:54.188461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:54.189932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:54.288863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 220/452] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:45:54.295286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:54.391000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434a56d-b81c-4f64-a220-5a5664a158f2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:54.416402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:54.424752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:54.425383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:54.425661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:54.425966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] PATH_INFO: `/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:54.427079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] RBAC: Authorizing `identity:get_project(project_id=dab10ae977dc434f9d22b1fb83ca19e8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:54.428516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:54.430155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 233/453] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:45:54 2022] GET /identity/v3/projects/dab10ae977dc434f9d22b1fb83ca19e8 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:54.435964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:54.506840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:54.531018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02b3c6-99ba-4990-ba84-62b819d2664d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:54.564403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:54.565015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:54.565292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:54.565564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] PATH_INFO: `/v3/groups/304d464f2de44165b83617a194129eac` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:54.566681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] RBAC: Authorizing `identity:get_group(group_id=304d464f2de44165b83617a194129eac)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:54.568123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08f9b5-022f-4c37-9e12-46f23817c4db None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:54.569652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 221/454] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 07:45:54 2022] GET /identity/v3/groups/304d464f2de44165b83617a194129eac => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:54.576297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:54.670867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d427cef-8fda-4e80-b74e-f34b9291109d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:54.703157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:54.703744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:54.704019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:54.704280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:54.704987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] RBAC: Adding query filter params (group.id=304d464f2de44165b83617a194129eac, role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=dab10ae977dc434f9d22b1fb83ca19e8) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:54.705289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:54.706738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393c119-e94b-4fd5-b908-f28e7e381b9f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:54.712820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 234/455] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 07:45:54 2022] GET /identity/v3/role_assignments?group.id=304d464f2de44165b83617a194129eac&scope.project.id=dab10ae977dc434f9d22b1fb83ca19e8&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:45:57.300375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47978-3f4a-4ecb-988b-7c0026d362c6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.300912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47978-3f4a-4ecb-988b-7c0026d362c6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.300912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde47978-3f4a-4ecb-988b-7c0026d362c6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.302551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 222/456] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:57.308696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668b2ba-5e4b-4e86-8285-a74ab169284c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.309154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668b2ba-5e4b-4e86-8285-a74ab169284c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.309252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668b2ba-5e4b-4e86-8285-a74ab169284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.334784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d668b2ba-5e4b-4e86-8285-a74ab169284c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:57.335991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d668b2ba-5e4b-4e86-8285-a74ab169284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:57.380595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bdb06-950e-4c9c-a74d-d325ab97b43b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.381134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bdb06-950e-4c9c-a74d-d325ab97b43b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.381569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bdb06-950e-4c9c-a74d-d325ab97b43b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.383340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 223/457] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:57.388779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad05d-d03b-49db-8cbb-ec60a4b8359c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.389055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad05d-d03b-49db-8cbb-ec60a4b8359c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.389520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ad05d-d03b-49db-8cbb-ec60a4b8359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.403653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:57.420195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d53ad05d-d03b-49db-8cbb-ec60a4b8359c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:57.421391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d53ad05d-d03b-49db-8cbb-ec60a4b8359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:57.435597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 235/458] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:57.528002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 224/459] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:57.679421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517eb3f-f4ed-4a3e-ab55-222bf37cb678 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.679991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517eb3f-f4ed-4a3e-ab55-222bf37cb678 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.680483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517eb3f-f4ed-4a3e-ab55-222bf37cb678 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.682259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 236/460] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:57.688179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202291-2e09-405b-906d-7f042961c993 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.688636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202291-2e09-405b-906d-7f042961c993 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.689077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b202291-2e09-405b-906d-7f042961c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.713823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4b202291-2e09-405b-906d-7f042961c993 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:57.715182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4b202291-2e09-405b-906d-7f042961c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:57.774868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517eb3f-f4ed-4a3e-ab55-222bf37cb678 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:57.799727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 225/461] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:57.892209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b202291-2e09-405b-906d-7f042961c993 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:57.894902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:57.899599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517eb3f-f4ed-4a3e-ab55-222bf37cb678 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:57.971268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:57.971828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:57.972101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:57.972371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:57.991497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:57.994158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:45:58.016474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:58.017653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:45:58.018782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:45:58.018782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.018782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:45:58.029696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 237/462] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:58.031804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.093081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b202291-2e09-405b-906d-7f042961c993 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.113040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:58.144289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.145291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.145715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.146181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.157279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.159247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:45:58.176688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:58.177846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:45:58.178882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:45:58.178882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.178882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:58.179041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c265aaf-aa0a-4f9c-a798-24c0799adf0a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.181222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 226/463] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:58.182232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.224626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.227169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.227452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.227719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.228375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:58.228672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.230077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.250600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 238/464] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:58.252706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:58.257873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.308186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889d4c5-83c0-4ae1-828c-0451d11a5e37 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.350038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.350949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.351454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.351872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.364446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.366275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:58.367715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.370495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d36cc-e345-4bf6-b89a-adafcf006223 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:45:58.385295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:58.386434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:45:58.387345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:45:58.387345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:58.387345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:58.389266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 227/465] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:58.390259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.412685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.413286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.413560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.413853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.415731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:58.416032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.417520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.429574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 239/466] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:58.435298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.503657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b09b38-6a9c-491a-bf45-d7240c6529aa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.512242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:58.538531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a372b9-3437-4920-8a96-35043133bddf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.560382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.561538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.561997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.562328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.562989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.564633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.576981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.578445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.578445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.578445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.579059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:58.579254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.580662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989c7fc-e421-46f1-92c0-22ef27666448 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.602927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 228/467] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:58 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:58.602927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.609889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 240/468] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:58.611567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcbe4-96bc-4ae6-9b6c-7d848084045d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.611919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcbe4-96bc-4ae6-9b6c-7d848084045d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.612188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcbe4-96bc-4ae6-9b6c-7d848084045d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.613670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 241/469] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:58.623629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.624071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.624174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.645310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:58.646634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:58.684595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:58.735973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 242/470] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:45:58.738693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:58.754926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e0d5-a1d6-4393-b9b1-98bc3f8db8df None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.809427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.810472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.810581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.811031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.811686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.813243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0ab2b-42db-47d0-8bb1-ca634bdca9d5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.843464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 229/471] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:58 2022] POST /identity/v3/users => generated 248 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:58.892363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d8360-582f-468f-921b-f13a01e6e74f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:58.932103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:58.932721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.932993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.933259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.933949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:58.935402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58090bcf-d5e4-4c7e-8626-8db6b759c26c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:58.960548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 243/472] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:58 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:58.977825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef3f-f8e4-4346-bac0-815ac6ddbe41 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.978470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef3f-f8e4-4346-bac0-815ac6ddbe41 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.978987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecef3f-f8e4-4346-bac0-815ac6ddbe41 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:58.980733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 230/473] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:45:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:45:58.987860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:58.988597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:58.989106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.006975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:59.018476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:45:59.020115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:45:59.075861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecef3f-f8e4-4346-bac0-815ac6ddbe41 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.105536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:59.106141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 244/474] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:45:59.207875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecef3f-f8e4-4346-bac0-815ac6ddbe41 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:59.244212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:59.245204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:59.245681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:59.246177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.258149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:59.260414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:45:59.275946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:59.276863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:45:59.277698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:45:59.277698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:45:59.277698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.279485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 231/475] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:59.285595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.395413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91203e-5640-47a2-996e-12be0adf8a58 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:59.417979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.440742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:59.441362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:59.441638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:59.441944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.442596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:45:59.442897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:59.444274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:59.447471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:59.457916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 245/476] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:45:59.463798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.507684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:59.544531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d98cf-9825-4ebf-9048-080c620811a0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:59.609601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:59.610245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:59.610522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:59.610801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.611496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:59.612872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:59.623956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35271252-6236-47b6-a9ef-354c0be49741 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:59.634964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 07:45:59.634964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 07:45:59.634964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [parameters: {'id': 'b0f6ca9cc9734d5b9b44664b45092606', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 07:45:59.634964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:45:59.662556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:45:59.663483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 07:45:59.664365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 07:45:59.665265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:45:59.667425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:45:59.669297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'id': 'b0f6ca9cc9734d5b9b44664b45092606', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:59.671303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 07:45:59.673345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.683856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 232/477] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 07:45:59 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:45:59.683856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.692256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:59.692878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:59.693155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:59.693427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.694123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:45:59.697508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:45:59.735820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 246/478] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:45:59 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:45:59.838847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7651-05de-44ed-9d7d-9ac7dd9f7579 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:45:59.887597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:45:59.888274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:45:59.888549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:45:59.888816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:45:59.891080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:45:59.910042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1dbc834a-8cb7-4cc9-abce-39d6263b2626 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:45:59.915910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:45:59.918111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 07:45:59.920081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:45:59.925819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 233/479] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:45:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:45:59.926101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:45:59.995057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:00.054315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc492363-5ae9-418c-aa34-f1acb3c08729 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:00.088955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:00.089493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:00.089790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:00.090112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:00.090782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:00.091082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:00.092475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d71d3-ce02-480d-993c-5f373f21acf2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:00.100642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 247/480] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:45:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:01.296745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314317-a5e2-4b07-ba7f-805c11d11479 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.297757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314317-a5e2-4b07-ba7f-805c11d11479 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.298247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b314317-a5e2-4b07-ba7f-805c11d11479 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.300085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 234/481] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:01.306177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788c33-c3c2-461d-b2bb-e102055e76dc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.306989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788c33-c3c2-461d-b2bb-e102055e76dc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.307469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788c33-c3c2-461d-b2bb-e102055e76dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.326307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-04788c33-c3c2-461d-b2bb-e102055e76dc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:01.327808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04788c33-c3c2-461d-b2bb-e102055e76dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:01.408841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 248/482] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:01.692283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6d3f7-afd3-4383-9a8d-351ddd8715f9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.692928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6d3f7-afd3-4383-9a8d-351ddd8715f9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.693414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6d3f7-afd3-4383-9a8d-351ddd8715f9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.694993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 235/483] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:01.700233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e21c-3eff-4ce4-8d9d-754c5c91e698 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.700736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e21c-3eff-4ce4-8d9d-754c5c91e698 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.701212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e21c-3eff-4ce4-8d9d-754c5c91e698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.720934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-dc34e21c-3eff-4ce4-8d9d-754c5c91e698 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:01.722413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dc34e21c-3eff-4ce4-8d9d-754c5c91e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:01.777547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:01.805053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6d3f7-afd3-4383-9a8d-351ddd8715f9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:01.812668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 249/484] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:01.962397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cf7d2-41d5-47da-990f-23ed6dac40b2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.962397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cf7d2-41d5-47da-990f-23ed6dac40b2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.962397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cf7d2-41d5-47da-990f-23ed6dac40b2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.965599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 250/485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:01.969864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:01.970150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:01.970422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:01.977975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6d3f7-afd3-4383-9a8d-351ddd8715f9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:01.997716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:01.998905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:02.042426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.043116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.043391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.043653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.067494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:02.073361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:02.075556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.077484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:02.081720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 236/486] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:02.087336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.129878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 251/487] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:46:02.131871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.175353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:02.253796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73c084-78cf-4c21-b2e7-4c9191d52ba7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.306310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7384e51-b728-409a-81c6-9c345c183d29 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.311719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:02.318656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.319856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.320396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.320891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.321812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:02.322327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:02.323907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:02.338357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 237/488] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:02.342581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.381253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.382293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.382817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.383341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.401470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:02.408842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:02.409832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.410634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:02.420367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 252/489] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:02.426692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.486134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488982-2022-42ba-9d92-f648a3c3e0eb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.506860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:02.529569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.530766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.531246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.531734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.544580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:02.546372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:02.564254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:02.565215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:02.566081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:02.566081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:02.566081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:02.568386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 238/490] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:02.569602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.584695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f5a0b-909d-432c-8d67-4f349c465e49 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.624078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.624711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.624984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.625251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.625955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:02.626254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:02.627634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:02.635734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 253/491] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:02.636534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.668684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:02.683303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cace58-e01e-4a4d-92b4-98d32ac93bd9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.720031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.720681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.720955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.721220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.733097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e97a2e2b-a23b-46c0-82b7-5f3bb8a7d057 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:02.738709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:02.739612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:02.740416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 239/492] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:02.741702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cd43-42f2-4cb9-afaf-ae65c9479ca7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.741702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cd43-42f2-4cb9-afaf-ae65c9479ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.741702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cd43-42f2-4cb9-afaf-ae65c9479ca7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.742891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 240/493] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:02.743404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649cd43-42f2-4cb9-afaf-ae65c9479ca7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.755409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56e18d-a0d6-4a50-9570-b7a4405eb026 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.774679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:02.795663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.796265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.796540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.796811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.797447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:02.797767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:02.799130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a1b46-5ac6-4c18-8d81-6baef6501d97 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:02.812409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 254/494] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:02.817789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.817789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.817789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.835215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:02.836944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:02.857191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649cd43-42f2-4cb9-afaf-ae65c9479ca7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:02.901889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:02.902550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:02.902828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:02.903098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:02.915636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:02.917178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:02.922471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 255/495] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:02.923520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:02.931983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:02.932972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:02.933826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:02.933826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:02.933826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:02.933826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 241/496] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:02.934053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:02.956710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.028184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.052672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4eb579-a157-4641-8da8-38c4363192ed None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.052882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd86e7-e28b-44bf-880b-7f8d6f5aa52e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.088486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.089105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.089379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.089642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.093364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.094005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.094288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.094558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.095246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:03.095519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.096989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.097173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.098452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.105409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 256/497] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:03.106443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-33ff0d39-de3d-4680-af35-6517f947f3bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:03.110829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:03.111911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:03.113180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:03.116301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 242/498] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:03.117545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ade4b-d540-456f-b93b-06608d6623b3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.117849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ade4b-d540-456f-b93b-06608d6623b3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.118124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ade4b-d540-456f-b93b-06608d6623b3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.119349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 243/499] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:03.120033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ade4b-d540-456f-b93b-06608d6623b3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.198302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.219813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687fb9f-d2ce-4a7f-aa9b-152bb1818e37 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.225134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ade4b-d540-456f-b93b-06608d6623b3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.253098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.253701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.254018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.254286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.254925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:03.255219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.256589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.261004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.261624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.261931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.262200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.269400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 257/500] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:03.270219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.270890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.273159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.274768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-74fdeed5-fcaa-4a5f-9a13-bfd149626dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:03.289207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:03.290156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:03.290998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:03.290998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:03.290998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:03.290998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 244/501] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:03.291474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.291766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.292017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.309605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:03.311095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:03.371343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147a81-f51e-47f0-ba08-415a22ae750b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.376066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 245/502] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:03.376876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.405958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.406563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.406839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.407105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.407825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:03.408125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.409542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.427708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.450139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 258/503] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:03.450139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.505343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc141f-3b44-4cd8-8fc5-3c763754f014 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.529746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.544421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.544991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.545270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.545541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.558105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:03.562396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d44fb-0e26-42de-a58e-48b35b30c87a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:03.564503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:03.565377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:03.566480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:03.567091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 246/504] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:03.567566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.602497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.603257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.603412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.603651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] PATH_INFO: `/v3/users/glance` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.611497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.613327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:03.627371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:03.628302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 259/505] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:03.629205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.676399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c5d71-16e5-4fd6-9b75-6b9950bd4b64 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.687599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.709769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.710369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.710644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.710917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.711632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:03.711928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.713299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.730243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 247/506] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:03.731026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.731663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3262-790e-4c18-a3fb-656404868eff None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.766438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036b900-c833-4953-b249-739dffc89562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.767038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036b900-c833-4953-b249-739dffc89562 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.767312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036b900-c833-4953-b249-739dffc89562 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.767580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036b900-c833-4953-b249-739dffc89562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.772725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.828622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07798bc6-e340-4773-9460-fa9ed4b6cee9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.838840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036b900-c833-4953-b249-739dffc89562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.840309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036b900-c833-4953-b249-739dffc89562 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.861250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:03.861881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:03.862158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:03.862424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:03.863077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:03.863372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:03.864716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:03.873211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 248/507] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:03.874191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.895187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:03.940711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 260/508] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:03.941656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036b900-c833-4953-b249-739dffc89562 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:03.973785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d53d4f-cbd9-474c-9b9f-3a2595050e71 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:03.976062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.007434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.008053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.008328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.008594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.009308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:04.009605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.011061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.040070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036b900-c833-4953-b249-739dffc89562 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.046896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 249/509] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:04.047698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.074083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.074698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.074973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.075237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.084874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.086810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:04.101005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:04.102525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:04.103370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:04.103370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.103370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:04.103370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 261/510] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:04.103582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.113927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.147324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd345f9b-72ad-4b7e-afb0-cc5ca2b90387 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.181939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.182556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.182830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.183099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.190381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.191816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.201455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0355dba-f55d-4799-8128-6395aeb26a93 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:04.203064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:04.204098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 250/511] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:04.205001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.230163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.240311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.240933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.241208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.241476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.251964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.253430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:04.267101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:04.268202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:04.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:04.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:04.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 262/512] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:04.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.311205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4417c-9abb-4a5f-9f75-238a42aa5592 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.329684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.344538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.345158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.345439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.345707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.356307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.357755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.370162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de6f9d-6c89-4270-a102-ddc41d799a93 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:04.371459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:04.372494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:04.373532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:04.373532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:04.373532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:04.373532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 251/513] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:04.373532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.407789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.408590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.408938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.409222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.447221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.472075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f8a1f-1838-4b3b-82ad-85535d45ea11 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.477079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.478562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.504906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.505512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.505819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.506092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.506726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:04.507032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.508867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.520208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 252/514] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:04.520985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.534585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.577004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 263/515] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:04.577950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.627306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074b8b-6fbd-41dc-81b9-eec0b80875d0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.661267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.661897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.662173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.662439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.663153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:04.663452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.664883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.666480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.675549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b874164-6ed6-41a0-925d-120800ab4f16 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.684081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 253/516] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:04.684887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.710063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.710671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.710953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.711253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.711889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:04.712193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.713559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.724651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 264/517] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:04.725406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.747887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.790173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22485ee6-35e5-40c8-b757-bdd1854e2051 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.824072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.824674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.824951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.825220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.825384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1b096-694f-4c3a-8e24-51a4cbb3c011 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:04.826025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:04.826331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.827742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.838791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 254/518] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:04.839664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.863805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:04.864402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:04.864674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:04.864943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:04.865584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:04.867046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:04.892469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 265/519] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:46:04 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:04.893286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:04.893619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:04.960360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b62e57-4640-4fb7-86c7-0aff37af1714 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.000796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.001603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.001969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.002258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.011041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.014593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.016109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:05.032210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:05.033368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:05.034265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:05.034265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:05.034265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:05.034265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 255/520] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:05.034265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.047337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a4505-4afa-4e6e-9bef-bb89b448712a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.087694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.088604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.089056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.089503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.150975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893b762-1093-44f4-9b7d-1e7010f51209 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.153997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.173648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.175184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.188438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.189067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.189343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.189610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.197638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.199130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:05.210806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:05.211734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:05.212629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 256/521] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:05.213214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.266623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.298007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 266/522] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:05.298007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.338829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc660cf-9fb6-49df-94fa-4d3a557c05ce None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.375473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.376104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.376404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.376650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.384409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.385720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:05.398088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:05.399494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:05.400378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 257/523] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:05.400845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.415960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a4a84-89a1-475a-99c3-9b3a1833c7e8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.429528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.452400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.453394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.453851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.454127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.454769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:05.455075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.456454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.473613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 267/524] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:05.473613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.510751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87aac8-1caa-4577-98be-9168849002b1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.513068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.549060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.549674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.549979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.550255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.550983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:05.551288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.552736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.570399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 258/525] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:05.571212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.592981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cf10e-93d2-4164-9556-e3719c3fa3cf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.628950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.629613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.629945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.630218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.641918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.643460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:05.658182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:05.659171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:05.660025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:05.660025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:05.660025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:05.660025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 268/526] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:05.660971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.668306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.688778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a889a4d-2edd-464a-8bb4-143dcf3fc7a3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.725576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.726234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.726565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.727409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.728101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:05.728404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.729855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.751889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 259/527] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:05.751889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.762909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.779472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c333a4-8d04-4d14-8632-01a7f3379fc0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.817922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.818556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.818830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.819097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.826718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.828150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:05.842336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:05.843189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:05.844061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 269/528] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:05.844538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:05.858428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c97ca7-5e76-4a86-a4ee-f18f8dffee6c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.897002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:05.897618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:05.897925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:05.898192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:05.910976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:05.965117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e86b24-7795-4242-b012-da2363b97112 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:05.970919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:05.972373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.000317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.000922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.001305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.001572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.002346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.002641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.004006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.016826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 270/529] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.017461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.028872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.079843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 260/530] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:06.080692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.126428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.127808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236697-c47a-4411-a1e0-123fc508bc02 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.163565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.164258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.164537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.164804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.165580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.165918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.167362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.173995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 271/531] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.182807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.189346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676041ef-af9e-45de-a1f8-3043f7b8e478 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.235761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.236714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.237141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.237572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.238552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.239020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.240965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.261434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 261/532] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.262790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.268122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.302077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ae373-2f65-4072-b119-ea33f029320d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.340212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.341033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.341644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.341779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.349203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.351069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:06.366119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:06.368168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:06.369526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:06.375022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 272/533] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:06.376039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.392045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dd651-bdac-4d02-aca4-1d83fb744af6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.393897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.440090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.441046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.441524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.442023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.453800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.455512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:06.470649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:06.471565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:06.472387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:06.472387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:06.472387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:06.474166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 262/534] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:06.475189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.499862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ed64-5f77-4f42-b44d-de11c8d5fcfb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.546177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.547770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.549221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.549505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.551287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.552774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=4ec0de2bbadf4dbd9a5b66992b382c17) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.554379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.557372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.565282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 273/535] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/role_assignments?user.id=4ec0de2bbadf4dbd9a5b66992b382c17&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.571190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.606346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be897ed6-1e8e-4a5a-8c5b-4902915227aa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.693372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.694069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.694347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.694616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.695364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.695676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.697668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.723804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 263/536] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.732648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.764030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:06.767587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27933f-07d6-4b1c-b3f7-3f009d809288 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.842019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:06.844836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:06.845377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:06.846150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:06.848105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:06.848618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:06.850921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:06.877712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 274/537] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:06.885293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:06.931444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c2c7a-e0ec-44d9-a554-459d1df67ca2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:06.970711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:07.005277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:07.006210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.006630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.007073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.008034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=c8e54e867eee421b8f230c259d3cce68) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:07.008752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:07.010426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:07.025042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 264/538] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/role_assignments?user.id=c8e54e867eee421b8f230c259d3cce68&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:07.043919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bde76d-9d19-4aa7-9fec-3ee65bae1dc2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:07.083520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:07.084432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.084858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.085331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.094348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:07.096164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4549c372-f92e-439f-9c94-53acd377d890 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:07.109857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:07.110948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:07.111989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:07.114128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 275/539] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:07.122879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:07.159227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ade24e-ac3c-4751-88de-d00cdf417457 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.159781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ade24e-ac3c-4751-88de-d00cdf417457 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.160258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ade24e-ac3c-4751-88de-d00cdf417457 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.161889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 276/540] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:07.171162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.171680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.172111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.192236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:07.193708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:07.281632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:07.293759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 277/541] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:46:07.307110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c586b-e49e-4f5c-ba2f-35ab5aa673f3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:07.342942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:07.343560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.343838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.344112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.344800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:07.345101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:07.346555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:07.359327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 265/542] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:07.364992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:07.414245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:07.470341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23744e-7f69-4602-8c1e-27afd02eeee4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:07.507614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:07.508879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.509165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.509435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.510200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=9e0abf2c4f1a45b88cf4f54237eb0def) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:07.510506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:07.511996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:07.519663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 278/543] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:07 2022] GET /identity/v3/role_assignments?user.id=9e0abf2c4f1a45b88cf4f54237eb0def&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:07.688281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:07.814276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76981d23-34cb-45bc-9240-c9650e178adc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:07.850213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:07.851176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:07.851705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:07.852201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:07.859204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:07.862701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:07.868189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:07.869090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:07.870364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 266/544] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:07.878831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:08.034332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada96d30-1a87-422f-ad4e-5e9eac465be2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:08.045641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:08.070642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:08.071252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.071533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.071804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.072468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:08.072768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:08.074177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:08.082728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 279/545] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:08.088538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:08.209015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f92b-be84-4b9f-9b61-ab5fd5649c22 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:08.245110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:08.245762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.246047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.246317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.256520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:08.258026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:08.272784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:08.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:08.274613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:08.274613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:08.274613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:08.274613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 267/546] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:08.282982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:08.411714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a7884-0ee7-485f-8393-c00d40841528 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:08.455566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:08.456182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.456469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.456741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.457449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:08.457773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:08.459215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:08.477922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 280/547] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:08.486815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:08.562154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:08.614433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4911819-1ca6-4ce5-a396-22e11ad0fab8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:08.649133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:08.649856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.650137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.650404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.658237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:08.659672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-dfb5e448-c80d-4bed-9711-7508d4fda35d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:08.671857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:08.673241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:08.674636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 268/548] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:08.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:08.686696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113afe1-271e-4cea-aff8-14f4577b1110 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.686985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113afe1-271e-4cea-aff8-14f4577b1110 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.687256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113afe1-271e-4cea-aff8-14f4577b1110 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.689823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 269/549] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:08.693939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.694224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.694500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.727324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:08.728537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:08.738710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:08.850065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528c493-b53c-4675-a9d3-c08079844c24 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:08.868882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 270/550] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:46:08.922520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:08.923060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:08.923338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:08.923607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:08.924340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:08.924642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:08.926102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae43f45-eddb-44b8-8cc0-4795c2166397 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:08.949374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 281/551] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:08.950890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.024320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.102869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d77b88-baf9-4920-864f-6f57b77bbd4a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.156281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.156901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.157191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.157463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.195041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04524-47e7-467f-b796-f46f79c5aa1c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.196301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04524-47e7-467f-b796-f46f79c5aa1c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.196773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04524-47e7-467f-b796-f46f79c5aa1c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.199625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 282/552] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:09.204157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.204440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.204714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.223898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:09.225079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:09.245689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:09.247176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:09.301092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.301707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 283/553] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 07:46:09.302851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.350000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.374192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 271/554] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:09.376392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.486042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98a02-ee8c-422c-9deb-5c730e25b5c1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.535828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.536458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.536735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.537061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.550233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:09.551056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0d823-93ad-48f8-9334-7fa1c2ab1118 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:09.555806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:09.556725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:09.557620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 284/555] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:09.566584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.633885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.635149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.635628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.636096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.645524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.648571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:09.651086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:09.659344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:09.667368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:09.668384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:09.669277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:09.669277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:09.669277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:09.671678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 272/556] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:09.672841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.685046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad0de-2579-4285-8841-1a89ca0765b7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.735915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.736585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.736805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.737074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.737770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:09.738055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:09.739596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:09.753544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 285/557] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:09.753544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.799528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959633be-0417-48cf-b6e5-795dee7563e4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.837179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.838007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.838255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.838470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.849067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:09.855042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:09.858399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 273/558] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:09.859223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.869178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3188c8a-e3d4-42a4-9be3-59a180edf0a1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:09.909552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:09.910427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.910572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.910727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.911369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:09.911670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:09.913043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acfcef-d673-468a-9225-3130af83c947 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:09.922610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.926254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 286/559] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:09.931969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c887d-646c-49ff-9817-4e7f90859f0f None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:09.932258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c887d-646c-49ff-9817-4e7f90859f0f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:09.932551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c887d-646c-49ff-9817-4e7f90859f0f None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:09.933777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 287/560] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:09.934473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c887d-646c-49ff-9817-4e7f90859f0f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:09.937434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:09.977668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4972-3072-4849-9baf-a4b416b07677 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.013101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.013883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.014225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.014566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.026158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.027598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:10.043775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:10.044739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:10.045603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:10.045603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:10.045603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:10.045603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 274/561] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:10.046453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.051059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c887d-646c-49ff-9817-4e7f90859f0f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.086578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.087188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.087468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.087735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.088397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.088696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.090093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bea11-a400-4bfc-b82a-ef2987f4bd62 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.103438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 288/562] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.103438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.103438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.103438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.118621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:10.119890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:10.153190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aa584-c4ca-498d-b4db-c49d0ce4b9e4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.156486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.183301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.191429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.192067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.192347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.192617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.193141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 289/563] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:10.193896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.202758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.204503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:10.221128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:10.222032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:10.222924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:10.225415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 275/564] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:10.226987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.306692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afc4aa-d538-4370-801a-f5e84e4f10c1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.352394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.353699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.354233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.355164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.356317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.357107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.359476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.360670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15cae9-bfa9-4dd9-98fd-e64985aa3bdb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.375723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 290/565] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.377095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.397282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.398086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.398350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.399167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.410508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.411885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:10.426409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:10.427365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:10.428254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:10.428254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:10.428254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:10.432236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 276/566] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:10.433096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.448873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.454713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.495987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc963980-c797-42a3-9de7-2eb0aacd9f36 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.531969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.532574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.532849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.533116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.533832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.534137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.535555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.539020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f893-d2d3-4cc4-80f5-67426a5141a6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.553314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 291/567] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.554153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.576385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.576953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.577236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.577510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.584962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.587142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9435cdb6-b307-4764-9611-f53236baf72a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:10.598142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:10.599221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:10.600354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 277/568] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:10.601071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b43d00-553f-4470-94a1-b0b004cac3f8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.601071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b43d00-553f-4470-94a1-b0b004cac3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.601312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b43d00-553f-4470-94a1-b0b004cac3f8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.602641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 278/569] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:10.603233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b43d00-553f-4470-94a1-b0b004cac3f8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.659105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647cdb5-2c46-4afc-8bac-b85af9041e79 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.673888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.678548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.695947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.696555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.696829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.697091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.697724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.698045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.699411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.703647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b43d00-553f-4470-94a1-b0b004cac3f8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.710127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 292/570] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.710942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.741324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.742506 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.742787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.743056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.753693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:10.758987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:10.760084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:10.761071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 279/571] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:10.762847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.807086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dfa73-0641-4293-8531-db4e701751b1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.840487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.841021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.841302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.841581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.842337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=89e32c33e2184f5c93cc526f085c5375) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.842649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.844075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8216c2-1167-4c02-be0f-46d26358f86b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.849291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 293/572] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/role_assignments?user.id=89e32c33e2184f5c93cc526f085c5375&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.850564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.850726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.850980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.863267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c4b0-80bc-4e61-a574-c20a75a13411 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:10.871602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:10.872854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:10.904590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:10.905229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:10.905513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:10.905810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:10.906655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:10.906878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:10.908391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:10.911862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.914600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:10.930694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 280/573] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:10.931424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:10.963975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 294/574] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:10.965128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.059516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331aa01-d55e-48de-8cdf-30d18837825b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.097883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.098448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.098728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.099037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.107532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.108079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477214ee-2d9d-4033-bb73-1acc78fc7ac4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.108980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:11.122189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.123177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:11.124123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 281/575] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:11.124711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.148623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.149603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.150133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.150662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.151638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:11.152162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.153899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.163170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 295/576] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:11.164763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.184867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.200106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.235772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b01bf-5dae-4844-844d-9495c4de9c60 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.271767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.272399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.272678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.272949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.275782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d61f06-5fd7-4c52-92aa-4d1daf685159 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.283668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.285109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:11.299582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:11.301415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:11.301415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.301415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:11.301415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 282/577] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:11.301647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.318234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.318882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.319163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.319446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.320342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:11.320643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.322304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.339736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 296/578] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:11.340396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.414864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcc179-5988-43b5-9c32-cea25fd7394a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.422768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.437502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.450630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c43f-c6b8-4499-9e6b-7bfe1f8ab9d5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.455039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.455667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.455946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.456215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.468224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.469669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:11.484346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.485487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:11.486410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:11.486410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.486410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:11.486410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 283/579] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:11.486631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.492471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.493112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.493393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.493670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.507303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:11.512369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.513261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:11.514129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:11.516686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 297/580] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:11.517688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.598311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90db69-3744-498b-b7ec-786e16f2b472 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.632117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe847-5953-4092-a271-c2348aea8abf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.634390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.635283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.635283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.635535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.636172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:11.636471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.637879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.651222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 284/581] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:11.656166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.671849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.672415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.672692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.672960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.675393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.684894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.686589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.696049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:11.702987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.703942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:11.704859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:11.704859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:11.704859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:11.704859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 298/582] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:11.705714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.774640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80ebb8-48a2-414a-8c0a-e723003e9081 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.809159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.809922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.810302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.810936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.811659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fea4e-2f4b-4c55-877f-199601e6e13a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.811759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:11.812130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.813614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.825925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 285/583] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:11.826920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.853708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.854650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.855157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.855633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.856477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:11.856961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.859247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.868840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 299/584] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:11.870159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:11.911622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.934773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63aabe-777f-4884-9053-cb918c6f22ea None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.950632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:11.971384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:11.972235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:11.972585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:11.972927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:11.980739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:11.982289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:11.992927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7e46-cac6-49a8-8d0a-f76914b0d513 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:11.993478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:11.994566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:11.995690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:12.000953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 286/585] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.000953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.037342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.038120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.038492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.038915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.039661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:12.040031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.041482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.057789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 300/586] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:12.057789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.106460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f6390-efd1-455f-9c95-e547cb0d91c2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.140818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.141452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.141767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.142035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.150105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.151557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.153423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.164711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef240b3-8589-425e-8596-553e8683cf36 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:12.168575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:12.169494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 287/587] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.175384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.189024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.204073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.205086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.205387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.205843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.216162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.217618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:12.232529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:12.233495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:12.235245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:12.235245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:12.235245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:12.235245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 301/588] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.235245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.281349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebd8af-ccab-4f68-bf46-509ba345d4f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.317775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.318616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.318889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.319505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.320816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:12.321425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.322904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.339516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 288/589] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:12.340356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.345212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf06b24-addc-42e4-a6d3-5e6b3d30b054 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.380582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.381209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.381488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.381785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.389171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.390667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:12.403178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:12.404111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 302/590] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.405369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.407719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.453420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.454743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2be44-f815-43fb-a08b-610fd42cf71c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.491535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.493208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.493208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.493407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.494052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:12.494405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.495885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.515409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 289/591] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:12.516479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.528163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02791d-2753-4eeb-a505-f445545f6f71 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.566511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.567184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.567459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.567728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.568417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:12.568718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.570135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.582403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 303/592] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:12.583227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.625788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d083cec-ee85-4c43-ab07-13084ca684b4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.642627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.661576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.662310 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.662590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.662861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.682059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=742f83bd38b542d193a911ef623abd45, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.682746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.684275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.690557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff411b-05ac-4b1a-9028-17d8d9cbf41e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.700992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 290/593] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:12 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:12.706790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.741279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.742647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.743104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.743552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.744475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:12.744966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.746638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.762725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 304/594] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:12.764063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.852304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec075-71b8-488f-b38b-9d6658bc0b4c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.893574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.894375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.894722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.895067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.911289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e4aec-b721-4771-b995-f458596bc7bf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:12.911610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.913350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:12.934500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:12.935761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:12.936647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:12.936647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:12.936647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:12.936647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 291/595] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.936871 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:12.957903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:12.958746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:12.959329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:12.959731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:12.960105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:12.970498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:12.971079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:12.984014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:12.984971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:12.985953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 305/596] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:12.987636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.004694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.058647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb426-821a-4727-90e2-55dd4b10215a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.097261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.098495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.099272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.099883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/4ec0de2bbadf4dbd9a5b66992b382c17/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.102081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e8756-c2f9-4bf0-b854-4529253450bc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.120454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=4ec0de2bbadf4dbd9a5b66992b382c17, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.122212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.137917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 292/597] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:12 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/4ec0de2bbadf4dbd9a5b66992b382c17/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:13.138735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.147376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.149156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.149631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.150100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.151633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:13.152249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.154813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ed02d-73d0-479f-9aa5-52953715d17c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.178986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 306/598] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:13.182349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b397a-6782-4601-bd10-a11f3f51fc8e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.184673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b397a-6782-4601-bd10-a11f3f51fc8e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.184957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b397a-6782-4601-bd10-a11f3f51fc8e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.187166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 307/599] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:13.188261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b397a-6782-4601-bd10-a11f3f51fc8e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.229943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.280205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776c41d-dcfd-4b28-8418-f87490c46152 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.282074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.317621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b397a-6782-4601-bd10-a11f3f51fc8e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.337240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.337929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.338379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.338658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.339716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:13.340023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.341464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7ee39-81e0-49f4-ae02-8bd320efcac0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.360641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.361776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.362151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.362515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.363325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 293/600] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:13.366154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.366154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.366154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.376616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.380021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.384320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:13.385530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:13.401928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:13.403021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:13.403986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:13.405827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 308/601] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:13.406750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.480186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 294/602] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:13.482362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.561506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114faf7-66ff-4af8-90ba-5d825fcaca0f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.589693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.625839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.626720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.627150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.629581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.634034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.653112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.654621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.669724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9652e-1edb-44b2-81a0-393e5b9e74c8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:13.675200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:13.676133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:13.677031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:13.677031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:13.677031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:13.679088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 309/603] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:13.686498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.714188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.714813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.715091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.715379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.716122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:13.716433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.717905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.734357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 295/604] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:13.739286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.867525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cd07a-de9d-4f79-a1fc-ea5404ae6a26 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.884256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:13.884753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4039f-827e-430a-b109-b79a63307f4c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:13.906550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.907448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.907862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.908206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/c8e54e867eee421b8f230c259d3cce68/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.924221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=c8e54e867eee421b8f230c259d3cce68, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.925831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.930287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:13.931260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:13.931805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:13.932045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:13.933782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:13.933782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:13.934649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:13.945491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 296/605] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:13 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/c8e54e867eee421b8f230c259d3cce68/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:46:13.946812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:13.962776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 310/606] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:13.964161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.056308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:14.080446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976091-d19f-48cc-866d-c8d28684a404 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.133501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.134181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.134459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.134727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.142898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7a483-8232-405b-952c-61b4ac6e2543 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.155518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:14.167325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:14.168341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:14.169216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:14.169894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 297/607] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:14.175817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.184802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.185701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.186012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.186288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.194187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.196224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:14.209448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:14.211930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:14.213144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 311/608] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:14.219757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.249341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:14.342259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934aca9-e97e-4b14-9149-4d9ae46ec8c9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.355638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffacf2-0a86-4640-9a60-7cef45aadb98 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.363668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:14.392313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.392929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.393206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.393470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.394178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:14.394478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.395872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.400244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.401179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.401699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.402194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.403135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:14.404305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.406335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.407217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 298/609] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:14.415418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.421121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 312/610] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:14.426437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.546265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c7476-78a7-4a89-bbce-0ea873f43304 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.588459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.589501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.590010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.590511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/9e0abf2c4f1a45b88cf4f54237eb0def/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.608461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=9e0abf2c4f1a45b88cf4f54237eb0def, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.610284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.626453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 313/611] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:14 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/9e0abf2c4f1a45b88cf4f54237eb0def/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:46:14.629579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:14.648279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9378dc-4c7c-4cdc-aa55-c186095ac1cd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.693619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.694340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.694619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.694959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.708357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.709865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:14.730041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:14.731054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:14.731992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:14.731992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:14.731992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:14.734115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 299/612] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:14.743504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:14.769455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:14.877326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5ac88-cb1f-4e2a-8267-876b387ebe6b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:14.918820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:14.920291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:14.920782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:14.921272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:14.922170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:14.923803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:14.925579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f12a0-65a2-422b-9ea3-0f0585953cd7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:14.939756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 314/613] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:14.944761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.100733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b765a0-e858-4ffe-979c-b2ec904deab1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:15.145774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:15.146615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.146993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.147377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.156193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:15.157791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:15.160399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:15.173663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:15.174727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:15.177100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:15.178480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374071-6ae6-4079-81f2-138d010d73af None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.178634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 300/614] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:15.178773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374071-6ae6-4079-81f2-138d010d73af None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.178773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e374071-6ae6-4079-81f2-138d010d73af None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.178773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 315/615] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:15.182529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.186944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.187231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.187650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.207348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:15.208846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:15.336344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 316/616] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:15.366450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4d61-41b7-43e9-9492-5a1757a35845 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:15.405163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:15.405693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.405992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.406262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.406985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:15.407284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:15.408697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:15.436201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 301/617] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:15.441470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.461186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:15.584471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dbd28-d62b-4278-88a0-01e96c4cd9dc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:15.625179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:15.625839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.626117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.626384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.637547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:15.639056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:15.656159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:15.665504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:15.667534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:15.669720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:15.669720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:15.669720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:15.669720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 317/618] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:15.672061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.764411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:15.766339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:15.812594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64fd1-1d6a-4e28-a874-b74afb15ff09 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:15.858508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:15.859126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.859406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.859678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.860335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:15.860637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:15.862091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:15.863981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0396e-9801-4218-a3f7-1b77c748cce3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:15.885435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 318/619] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:15.890399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:15.941779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:15.942338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.942623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.942893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.958553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-182972d8-58af-40bd-bba5-73b02a6d8a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:15.965370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:15.967336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:15.968399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 302/620] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:15.969690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da88192-9659-4352-b91a-0cb057c3cb51 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:15.969690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da88192-9659-4352-b91a-0cb057c3cb51 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:15.969690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da88192-9659-4352-b91a-0cb057c3cb51 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:15.974867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 303/621] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:15.975593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da88192-9659-4352-b91a-0cb057c3cb51 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.017592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf7483-f0f7-43af-bf4b-cc1f829d2add None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.036527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.055573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.056299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.056647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.056988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.064433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.065986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d836da2c-2855-4410-bebb-6a3f84e233f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:16.078122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:16.079269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:16.080383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:16.085682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 319/622] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:16.087182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.087535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.087892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.106573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:16.108098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:16.111834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da88192-9659-4352-b91a-0cb057c3cb51 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.130320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.149210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.150150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.150670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.151129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.151969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:16.152455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.154081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.164340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 304/623] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:16.165597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.189760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 320/624] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:16.189760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.305605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c74ea-2cb0-4ba7-8f1f-44b52f4ac34a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.349580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.350706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.353420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.353959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.355036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:16.356108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.357938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.371337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 305/625] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:16.372641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883d01d-0779-44e7-b350-1b632656c5c6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.378666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.388568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.432318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.470989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.471644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.471925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.472195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.490730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d2b3-ccec-4ae0-821a-ab4ebe938117 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.502769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.504227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.530309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.531395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.531879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.532402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/89e32c33e2184f5c93cc526f085c5375/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:16.539530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:16.540611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:16.541560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:16.541560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:16.541560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:16.545211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 321/626] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:16.546251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.555140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=89e32c33e2184f5c93cc526f085c5375, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.557041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.575077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 306/627] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:16 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/89e32c33e2184f5c93cc526f085c5375/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:16.576624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.704765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a675ca-2e4f-4bfd-a0be-a4a2dbb49560 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.724432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.739531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a6d2a-a82c-4523-bb99-2a4351f564ef None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:16.756970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.757666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.758048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.758324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.773720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-cc2d7be3-df89-4e2b-b403-219b255e7562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:16.781151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:16.782430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:16.783577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:16.784328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 322/628] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:16.785196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e177-9fe4-4605-8fda-23e0dfd67019 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.785475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e177-9fe4-4605-8fda-23e0dfd67019 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.785764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e177-9fe4-4605-8fda-23e0dfd67019 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.787134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 323/629] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:16.790975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52e177-9fe4-4605-8fda-23e0dfd67019 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.792330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:16.792943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.793502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.793808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.794476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:16.794860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:16.796153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02be31-66d5-4ff7-ad35-8b24888558ee None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:16.811045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 307/630] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:16.811397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:16.811978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:16.811978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:16.822102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:16.834226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:16.835949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:16.916020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 308/631] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:16.917182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:16.942466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52e177-9fe4-4605-8fda-23e0dfd67019 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.005072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.006579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.007263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.008815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.010243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:17.010883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.013242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.030815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 324/632] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:17.035812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.054993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.093051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215eddb-d730-4f8d-9632-7eb1b72a2c73 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.162477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.163403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.163872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.164285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.178107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.181962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.183477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:17.206981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:17.208049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:17.209674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 309/633] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:17.214754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.217019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccac449-60ad-466d-babf-fa28754472fb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.264975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.265993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.266475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.266979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.280374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.282554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:17.299337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:17.300329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:17.301263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:17.301263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:17.301263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:17.303687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 325/634] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:17.304863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.348559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b698da-0b06-4ab9-ad7b-617cf076f601 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.387128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.387738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.388021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.388292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.389020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:17.389343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.390835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.402937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.412779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 310/635] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:17.413651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.420015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7669c16-ff27-406d-a94d-3e0b04e7c61f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.457688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.458401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.458680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.459162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.460893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.472372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c17bb3a1-36e0-42ca-a007-f4e30fe5c078 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:17.477681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:17.478661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:17.479503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 326/636] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:17.480779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600d4fd-a451-454a-ad07-0f55450cd490 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.480779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600d4fd-a451-454a-ad07-0f55450cd490 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.480955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600d4fd-a451-454a-ad07-0f55450cd490 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.486321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 327/637] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:17.487105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600d4fd-a451-454a-ad07-0f55450cd490 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.536940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1be16-1a7d-4813-a42d-06911f7072b8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.581682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.582705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.583179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.583653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.584485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:17.584980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.586969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.606230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 311/638] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:17.607014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.621529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600d4fd-a451-454a-ad07-0f55450cd490 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.665954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f38763-2989-4001-b546-b301d662212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.666569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f38763-2989-4001-b546-b301d662212b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.666846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f38763-2989-4001-b546-b301d662212b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.667119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f38763-2989-4001-b546-b301d662212b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.674430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.720560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f1a4-e4d6-4269-91b6-bdfca9bf2965 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:17.738366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:17.746656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f38763-2989-4001-b546-b301d662212b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.748114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f38763-2989-4001-b546-b301d662212b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.759845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:17.760596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.760934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.761271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.762153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:17.762519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:17.763955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cc281-6e62-4830-a599-a8a227998582 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:17.778031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 312/639] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:17.779421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:17.779875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:17.780219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:17.799313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:17.800567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:17.872098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 328/640] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:17.872927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f38763-2989-4001-b546-b301d662212b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.886278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 313/641] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:17.887594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:17.967436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.009970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.018501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f38763-2989-4001-b546-b301d662212b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.046070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f860264-b853-4490-8c3f-80c6a67e2b2f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.056899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.057372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.057481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.057897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.065599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.067818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:18.080384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.081313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:18.082288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 329/642] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.082887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.095980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.096743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.097113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.097534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.108832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.110461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:18.126629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.127841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:18.128927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:18.128927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:18.128927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:18.134549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 314/643] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.135443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.194703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5dc6-e6e0-4a56-8198-19ad41aac144 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.228267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.228894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.229174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.229444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.239782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.241243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.249228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138ec7a-73e9-427e-b9a6-ba7a4d44d38b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.250762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:18.255737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.256683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:18.257809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:18.257809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:18.257809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:18.261417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 330/644] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.262132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.271156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.288508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.289623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.289944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.290188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.290912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:18.291207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.292625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.314257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 315/645] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:18.314257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.376889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d659e-4e3f-4ff6-80ed-653caa7d9d69 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.411720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.412337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.412619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.412892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.414092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:18.414406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.415556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2f243-7cd8-4232-8cfc-0905635f2374 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.415775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.428430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 331/646] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:18.429310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.454135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.454774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.455042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.455310 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.467189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:18.472685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.473649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 316/647] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.478523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.498512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.500611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.535263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458335b9-1fd7-4d67-9ca0-fe9910a8a139 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.572043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.572656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.572933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.573208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.573867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:18.574169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.575544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.583551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbaa24-a992-4613-b7f5-36a6ecf89bd9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.591715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 332/648] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:18.591715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.621612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.622561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.622962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.623379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.695922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4cb16-dfcb-4f0a-90a0-ed85687abd28 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.698743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.700194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.721161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.726374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.735450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.736442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.736442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.736644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.744287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.745746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:18.758092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.759045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 333/649] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.759979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.813654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 317/650] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:18.813654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.876170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd03454-7ed2-4130-882e-d11fed5547df None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.912563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.913336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.913948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.914244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.915171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:18.915471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.916484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377fde-8974-4e09-aa8b-408f394dc478 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:18.916837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.925813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 334/651] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:18.926624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:18.938405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.953416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:18.954020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:18.954301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:18.954571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:18.961204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:18.962670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:18.973704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:18.974755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:18.975653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 318/652] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:18.976222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:18.976348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.030267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b00b26-9f5f-4e4c-bd8e-5de45788a461 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.065256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.065985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.066163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.066436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.067159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:19.067460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.068885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8225f-9381-4829-92f4-d4596ec27efe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.073969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a950c57-461d-4bab-a2ea-ffbb9aeb21b9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.085814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 335/653] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:19.086978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad6cc-b79c-4cc1-86d8-85427fc33caa None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.087259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad6cc-b79c-4cc1-86d8-85427fc33caa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.087527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cad6cc-b79c-4cc1-86d8-85427fc33caa None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.088868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 336/654] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:19.089384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cad6cc-b79c-4cc1-86d8-85427fc33caa None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.109135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.109781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.110065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.110335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.120807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.122275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:19.137126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:19.138294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:19.139367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:19.139367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.139367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:19.139367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 319/655] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:19.139367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.169108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.192192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cad6cc-b79c-4cc1-86d8-85427fc33caa None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.200495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.226416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.226972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.227241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.227504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.236703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2e66-a58b-41a8-bed0-fc6ccb39cf74 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.236945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.238421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4306d7ea-25ed-473a-8c70-5fd778bf3f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:19.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:19.254227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:19.255155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:19.255155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.255155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:19.255155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 337/656] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:19.255769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.255867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.256090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.273062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.273660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.273963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.274174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:19.274267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.274890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:19.275185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.275353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:19.276583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.288192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 320/657] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:19.289461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.342857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 338/658] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:19.343738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.380644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.389861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e97655-71d2-4276-9201-4e6eb9a78507 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.424782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.425526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.425943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.426323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.445288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.462791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7b25-557a-48d7-8b6e-4c1f78ade94c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.498667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.500047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.506083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.506802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.507228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.507704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.508447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:19.508871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.510475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.526673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 339/659] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:19.526673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.594041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.608665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 321/660] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:19.608665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.635209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de3f6d-392e-4b10-8bf5-d05aa5a74de3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.668828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.669442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.669722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.670028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.670676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:19.670981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.672370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.683901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 340/661] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:19.684699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.695752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.706916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2043f-f110-4e0e-b981-4eff930dec5a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.740293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.740888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.741164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.741428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.742199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:19.742499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.743910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.749613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 322/662] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:19.750282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.807097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cbda9-505d-4277-b741-210b9791f1c2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.813970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.846764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.847668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.847944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.848204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.856153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.857756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.866995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afddaf-b19a-49fc-b71a-7f29d4161d5b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:19.869453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:19.870541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 341/663] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:19.876314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.928125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:19.928812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:19.929198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:19.930188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:19.944340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:19.945910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:19.951352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:19.962750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:19.963740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:19.964626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:19.964626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:19.964626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:19.964626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 323/664] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:19.964626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:19.992172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e0b11-b019-4664-9729-c96995a4a363 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.030432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.031106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.031283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.031567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.043317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:20.048517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:20.049443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 342/665] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:20.051433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.084559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf82462-ce4e-4073-a28f-34f6af3a631a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.096704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.125875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.126518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.127476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.127476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.136268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.137595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:20.149693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:20.150668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:20.151775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 324/666] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:20.156125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.161584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76414122-aaad-4842-b712-64fa94e62a83 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.188831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.198265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.198879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.199164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.199441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.200137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.200446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.201956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.214941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 343/667] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.216706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.270639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd995ab-de45-450a-b542-beb64262cc98 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.306340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.306953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.307231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.307504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.308140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.308438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.309804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.322753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 325/668] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.323397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.330451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53dafd-9d02-49eb-9de1-7447ecd6e828 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.354375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.367682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.368021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.368294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.368562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.369216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.369569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.370982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.384569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 344/669] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.384569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.429932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946750a-5274-4583-93d0-d0be2ec86ad7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.437215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.465635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.466269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.466571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.466839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.467560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.467859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.469270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.488068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 326/670] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.488908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.506021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a9943-c705-4b62-a42b-7279a2395cf7 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.543949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.544572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.544874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.545214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.545988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=4ec0de2bbadf4dbd9a5b66992b382c17) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.546328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.547759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.553716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 345/671] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/role_assignments?user.id=4ec0de2bbadf4dbd9a5b66992b382c17&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.555247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.594888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.619716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c991b2f-f5ce-454c-85a8-57e16b9cd663 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.676030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.677024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.677493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.678671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.690974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.692448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e285ee2-d5df-463f-9720-bb4a556b4c3b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.692986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.710728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:20.713002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:20.714203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:20.715350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:20.724186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 327/672] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:20.724186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.740155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.740803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.741084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.741360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.755163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.756643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:20.772882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:20.773958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:20.774864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:20.774864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:20.774864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:20.774864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 346/673] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:20.775527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.855238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53d164-8e6d-4c75-9410-22d8999baf19 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.894379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04599ab8-ce36-43e9-ac2a-e98b0f672215 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:20.902652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.903280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.903558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.903833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.928564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:20.934147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:20.934691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:20.935011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:20.935537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:20.936418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:20.936726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:20.938201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:20.951514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 347/674] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:20.952351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:20.980958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.006168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.007672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.063576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c73d4-6816-4e67-a423-2b9d6507b3a8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.106825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.108653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.109223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.109688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.110549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:21.111021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.112571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.129814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 348/675] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:21.131652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.148719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 328/676] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:21.148719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.197893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.242898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.259599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea5222-a0b2-40d7-8b78-ad5a1a848acf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.268752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5591c-627d-4547-8608-56f735c31f6c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.295693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.296558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.296690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.296881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.297605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=c8e54e867eee421b8f230c259d3cce68) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:21.297942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.299490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.305686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 349/677] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/role_assignments?user.id=c8e54e867eee421b8f230c259d3cce68&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:21.306615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.314136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.315145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.315621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.316732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.327571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.329679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:21.346658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:21.347674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:21.348609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 329/678] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:21.354026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.433375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a67c5-fe0a-4465-8ea7-8e8b74e82336 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.476388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.477534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.478049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.478513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.495022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.498853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:21.531224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:21.532240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:21.533116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:21.533116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:21.533116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:21.537345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 350/679] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:21.542478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448c558-778c-4a26-8b1b-a0ee0a5a13dc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.551475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.554156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.557120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.588589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.590339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.590755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.591027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.591913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:21.592221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.594303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.616837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 330/680] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:21.622948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.679884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6a9c-ce26-49a2-9bd5-ce2acbbfca39 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.721380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.721988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.722271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.723191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.723869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:21.724179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.728335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.752266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 351/681] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:21.768325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c4caf-3846-4da6-9747-02288c2d7687 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:21.775733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:21.833093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:21.833704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:21.834011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:21.834281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:21.869927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.938138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:21.955584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:21.957349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:21.960170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246470fa-ce5f-4b9f-9f14-b1aa6da191e3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.005485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.006188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.006551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.006894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.015308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.016862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:22.029033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:22.030155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:22.031277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 352/682] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:22.036699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.079527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 331/683] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:22.086840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.170115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:22.194365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766150a-ed2c-43d6-bd8a-5badc3d1c384 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.212828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d89e0b-ed64-4899-963f-fe64b2d83ee2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.219979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:22.232591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.233469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.233926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.234347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.235183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:22.235809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.237388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319e77d-be19-4cb7-a3b0-7e4dbde16f7c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.250326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.251113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.251291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.251618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.251774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 353/684] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:22.253500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2fb45-12de-4fef-a2d3-f28a2ae353ab None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.253960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2fb45-12de-4fef-a2d3-f28a2ae353ab None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.254377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2fb45-12de-4fef-a2d3-f28a2ae353ab None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.260686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 354/685] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:22.261653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2fb45-12de-4fef-a2d3-f28a2ae353ab None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.268217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.269670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5a7ba992-c58f-4ef8-8807-e74c5ec02c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:22.285459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:22.286680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:22.287721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:22.287721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:22.287721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:22.291925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 332/686] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:22.293380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.293670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.294221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.313880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:22.315371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:22.383444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2fb45-12de-4fef-a2d3-f28a2ae353ab None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.402114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 333/687] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:22.403147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.425131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.426259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.426784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.427202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.428140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=9e0abf2c4f1a45b88cf4f54237eb0def) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:22.429465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.431608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.439679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 355/688] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/role_assignments?user.id=9e0abf2c4f1a45b88cf4f54237eb0def&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:22.497816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:22.577609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30897a-74fd-43f9-aa2c-cfcaf41fa1b8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.615713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.616940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.617175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.617445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.618130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:22.618431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.619906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.637754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 334/689] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:22.643522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.714472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:22.734674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.781921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff183e-dfe3-43e4-9895-cfb52874b094 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.830134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.830711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.830991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.831263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.840458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:22.842797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:22.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:22.860419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:22.861325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:22.861804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69db14-ed08-4d24-9f0e-66028901b0c3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:22.864963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 356/690] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:22.870219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:22.873866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:22.914218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:22.915899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:22.916672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:22.917147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:22.932915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:22.941042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:22.942100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:22.943035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:22.946983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 335/691] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:22.953167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.007911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cd285-456d-43b1-a752-ca12b68a90fa None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.017949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.078643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.079425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.079828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.080274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.080998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:23.081372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.082195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717c1e6-588a-4844-ae02-c75238674a31 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.083081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2fb10-831d-4c74-9e5d-d268aac7d718 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.096854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 357/692] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:23.103057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65352e-d719-419b-babb-10da8eeffae1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.103057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65352e-d719-419b-babb-10da8eeffae1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.103057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65352e-d719-419b-babb-10da8eeffae1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.103057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 358/693] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:23.105828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65352e-d719-419b-babb-10da8eeffae1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.130117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.130924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.131270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.131607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.132385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:23.132754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.134466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308064cb-60bc-4947-bfdd-702fdf958678 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.143985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 336/694] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:23.150296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.150712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.151062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.171043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:23.172263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:23.197179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.238198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65352e-d719-419b-babb-10da8eeffae1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.254027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 337/695] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:23.254995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.279748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.281057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.282520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.283495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.284747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=89e32c33e2184f5c93cc526f085c5375) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:23.286999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.289188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.301536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 359/696] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/role_assignments?user.id=89e32c33e2184f5c93cc526f085c5375&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:23.407995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cedbca-7ac5-4433-b74b-d226f4b422f1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.463255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.464225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.464712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.465163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.477493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.479254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:23.494935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:23.495920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:23.496862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:23.496862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:23.496862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:23.498645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 338/697] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:23.506821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.541347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.586454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.599969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.626548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935fa19-403a-4dd4-928b-59d927f165ca None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.669108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.670087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.670577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.671092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.671978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:23.672484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.674103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.688366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 360/698] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:23.696024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.707052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60b9c-4053-4661-9676-f900cf1cc400 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.755566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.756240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.756520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.756793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.774529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.776429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:23.797320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:23.798465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:23.799360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:23.799360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:23.799360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:23.799360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 339/699] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:23.805347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.826455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.841393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5181a-4344-4edb-9b70-5c930bc26aae None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:23.883891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:23.884546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:23.884860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:23.885114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:23.893345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:23.894579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:23.900151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:23.907412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:23.908398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:23.909414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 361/700] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:23.932489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:23.958780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97be78-f0a3-4e9e-8b98-bd092b0ce36b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.008605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.009580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.010184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.010827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.012112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:24.012979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.014655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.033040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 340/701] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:24.038980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:24.138272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472f807-97d9-4140-862e-e22c636352bc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.170927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255b7a0-9224-42f6-9888-b36fdc65f2c4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.187130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.188321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.188825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.189575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.191376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:24.192279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.195433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ca48f-07e4-4fa8-a7c8-0fdfc9e41df5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.202032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:24.218280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.219091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.220219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.220618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.221364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.223220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544fe21-95c1-4d8a-918c-05658578904d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.229627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 362/702] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:24.231289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed6f8c-e8e3-4281-817c-4f4127dda29b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.231741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed6f8c-e8e3-4281-817c-4f4127dda29b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.232157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed6f8c-e8e3-4281-817c-4f4127dda29b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.238220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 363/703] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:24.239900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed6f8c-e8e3-4281-817c-4f4127dda29b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:24.252747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:24.258500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 341/704] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:46:24 2022] POST /identity/v3/users => generated 254 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:24.260090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.260659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.261363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.285351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:24.289107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:24.353661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed6f8c-e8e3-4281-817c-4f4127dda29b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.379785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 342/705] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:24.389996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.390620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.390896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.391166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.466157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.467555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.479279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:24.576810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 364/706] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:24.582483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:24.695357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290429d-ff44-451c-a26f-78ad6b263dc1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.706409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:24.734957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.735513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.735789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.736055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.746621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.748066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:24.763499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:24.764507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:24.765399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:24.765399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:24.765399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:24.765399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 343/707] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:24.773766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:24.814795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:24.908692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ebc7a-410a-49c0-919d-56c434f5c69c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.924581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:24.937929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867a7ce-dd45-4cec-8072-d609d5b4fef9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:24.972309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.972926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.973206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.973480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.974148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:24.974452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.981755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.983658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:24.984638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:24.985185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:24.985660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:24.986521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:24.988098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:24.990705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 365/708] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:24.999120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:25.005719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:25.006868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:25.008233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:25.011077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 344/709] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 07:46:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:25.016970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.045199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:25.118341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbc9a9-5b0a-4723-9117-cc18def6f3e8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:25.156017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:25.156814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.157197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.157585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.168002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:25.168002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:25.181889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:25.182963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:25.184076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 366/710] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:25.190716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.199069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f928aa-a3d0-4474-a582-da422c58127c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:25.232521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:25.256359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:25.257044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.257904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.258335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.259131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:25.259524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:25.260966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a11390-e707-412b-8b5f-83873afea7ac None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:25.268620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 345/711] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:25.270369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf242da-ef9b-42b3-824e-4e7c62dae5e4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.271013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf242da-ef9b-42b3-824e-4e7c62dae5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.271359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf242da-ef9b-42b3-824e-4e7c62dae5e4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.273998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 346/712] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:25.274769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf242da-ef9b-42b3-824e-4e7c62dae5e4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.310809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d589fda-c9bb-4c03-8d61-02ba70fa9253 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:25.345583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:25.346438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.346811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.347180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.347966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:25.348365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:25.349933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6c20d-e610-4cae-a66c-7085bf83cf3a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:25.363230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 367/713] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:25.364679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.365053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.365437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.382153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:25.384317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:25.385683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:25.393219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf242da-ef9b-42b3-824e-4e7c62dae5e4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:25.431470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:25.432270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.432624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.432978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.433675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:25.434069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:25.435849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:25.441871 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 347/714] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:25.442778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.474069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 368/715] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:25.630584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24dd2-1247-42af-a7dd-4511004ffa06 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:25.669309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:25.669970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:25.670250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:25.670518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:25.671249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:25.671555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:25.672979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:25.679139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 348/716] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:25 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:25.818471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:25.841629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:25.959656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3c33d-fe4b-4352-8f46-6d0817d12b1f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:26.003274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:26.003912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:26.004186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:26.004451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] PATH_INFO: `/v3/services/network` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:26.005034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:26.006417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:26.017188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:26.018287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:26.019166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 369/717] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:25 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:26.024880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:26.044095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:26.161571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007fbfa-1c9f-49e7-ba7e-daf0b6ffbccb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:26.203427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:26.204472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:26.204953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:26.205466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:26.206306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] RBAC: Adding query filter params (name=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:26.206787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:26.208345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f7ba9-619e-4759-8b89-7325b3207800 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:26.215677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 349/718] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:26.220869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:26.362962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:26.383835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f12a-e514-475a-8f60-c9f2c8025d4e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:26.437174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:26.437712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:26.438031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:26.438301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:26.438939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] RBAC: Adding query filter params (type=network) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:26.439241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:26.440617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353a7b9-59a6-461d-b4eb-858d7a043b56 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:26.447978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f4085-72e8-44ec-b4f7-51229d6b2828 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:26.448891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f4085-72e8-44ec-b4f7-51229d6b2828 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:26.449369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f4085-72e8-44ec-b4f7-51229d6b2828 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:26.449475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 370/719] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:26.451018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 350/720] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:26.465358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:26.465713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:26.465930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:26.496365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:26.497594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:26.597436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 371/721] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:26.912540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f4085-72e8-44ec-b4f7-51229d6b2828 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:27.009530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:27.030635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f4085-72e8-44ec-b4f7-51229d6b2828 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:27.080288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:27.080923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.081197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.081472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.096309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:27.101676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:27.102671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:27.103563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 351/722] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:27.110498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:27.204345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:27.231051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb5739-b4eb-4cb9-ad50-0f6fab1d69b4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:27.282436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:27.283051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.283326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.283590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.284247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:27.284547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:27.285952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae31f28-00a8-417c-817c-1c6efe9182b6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:27.295092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 372/723] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:27.300748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:27.451360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f6390-10e2-4dc6-8359-4ec66b745cba None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:27.498723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:27.499288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.499569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.499844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.504292 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19447-48c6-4f10-a01f-241bb9096461 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.504825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19447-48c6-4f10-a01f-241bb9096461 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.504825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19447-48c6-4f10-a01f-241bb9096461 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.506460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 373/724] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:27.517605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:27.517792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.518077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.519121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.520714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:27.523075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:27.539097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:27.540095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:27.541034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:27.541034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:27.541034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:27.543355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 352/725] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:27.549015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:27.567589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:27.569835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:27.671945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 374/726] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:27.724183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fde5ee-a2ba-4da7-9b02-46e1be03a962 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:27.770768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:27.771741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.772877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.773350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.774242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:27.774870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:27.776495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:27.794290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 353/727] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:27.799521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:27.883335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:27.920345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d8339-00e6-4dea-945a-94345e3a3764 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:27.962590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:27.963235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:27.963573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:27.963851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:27.971333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:27.972786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:27.984864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:27.985888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:27.987384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 375/728] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:27.994498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.075281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.112052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.135688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768d14b-c675-4bfa-8fea-e6657b7dfcef None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.170915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.171938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.172390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.172826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.173770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:28.174268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:28.175856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:28.195573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 354/729] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:28.201678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.277110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c7bf-e629-4999-bdef-6a5a211ab7cf None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.315009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.326772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.328134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.328856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.329316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.340030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfdb5-310b-4aad-b789-739434a785e2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.344309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b27661c5-6083-4643-8fd8-577dbbe06509 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:28.351301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:28.352268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:28.353102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:28.354250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 376/730] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:28.354707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.355031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff4e-788d-4c5a-b551-7b23ed7c9e36 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.355312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff4e-788d-4c5a-b551-7b23ed7c9e36 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.355584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ff4e-788d-4c5a-b551-7b23ed7c9e36 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.357069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 377/731] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:28.359582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef0f9-4d37-47bb-a1cd-91556a2723d7 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.359862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef0f9-4d37-47bb-a1cd-91556a2723d7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.360132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ef0f9-4d37-47bb-a1cd-91556a2723d7 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.362710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 378/732] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:28.364338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.364928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.365374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.383065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.383677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.383953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.384220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.384774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:28.385968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:28.454768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:28.456235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d525c-c5d8-4016-8be7-96d0df528068 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:28.458454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 379/733] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:28.460287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.568972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996618d-8c8f-4aab-9cfc-0c8bea5f57f7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.593269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.597925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 355/734] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:28.599457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.599819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.600167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.619050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.619784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.620060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.620373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.621340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:28.621645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:28.623132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:28.625281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:28.626831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:28.630535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.633359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 380/735] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:28.634383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.717132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 356/736] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:28.718248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.778401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced55805-f150-480c-939e-22098d9d7e94 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.825114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.826237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.826742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.827330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.842786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:28.845385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:28.858818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df786c56-d616-4e63-92ed-284e72558c26 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:28.862028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:28.863039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:28.863950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:28.863950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:28.863950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:28.863950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 381/737] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:28.864185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.900478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.903473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:28.904578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:28.905132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:28.906044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:28.919148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:28.922298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:28.923949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:28.941624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:28.942602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:28.943493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:28.943493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:28.943493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:28.945724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 357/738] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:28.946797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:28.982422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d50c1-43cf-4808-ad8d-0fb49fd35f0c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.016042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.016645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.016919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.017189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.017866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.019230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.041696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 382/739] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:28 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:29.042820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.082530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938c39a-f945-42e8-99a7-0aa91d1a8323 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.150456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.151007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.151286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.154021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.155673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:29.156500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.158423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.159266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.172595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a3f47-8846-43cc-a8e6-94c68ca7f66f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.183837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 358/740] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:29.185983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.216149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.216776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.217055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.217328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.218049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:29.218356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.220233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.233169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 383/741] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:29.234241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.256274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.313126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2f1ab-16c0-450c-82cb-59d6ee59e64e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.348662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.349297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.349583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.349887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.352016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caa229-e5d3-45f8-9b32-e84b5a7074f9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.360541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-d5edf33d-fe6d-44f3-b5d8-e126e0787e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:29.365302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:29.366319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:29.367160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 359/742] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:29.368479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19e88-a9ca-4697-94fc-d0c687ddde86 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.368479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19e88-a9ca-4697-94fc-d0c687ddde86 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.368479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e19e88-a9ca-4697-94fc-d0c687ddde86 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.373799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 360/743] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:29.373799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e19e88-a9ca-4697-94fc-d0c687ddde86 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.395591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.396371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.396654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.396927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.404932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.406351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:29.420590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:29.421503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:29.422734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 384/744] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:29.423214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.439268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.482907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e19e88-a9ca-4697-94fc-d0c687ddde86 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.494670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.522410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.523041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.523318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.523663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.530796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.532219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.532323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f61ca9-8e32-45e4-87b1-18d4be021351 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-cbc64c42-4082-41c6-955a-2015cea0b3d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:29.544774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:29.545806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:29.547459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 361/745] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:29.548060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.548060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.548181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.567916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:29.569155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:29.576577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.577255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.577962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.578244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.578922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:29.579224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.581296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.590175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 385/746] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:29.591020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.662097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 362/747] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 07:46:29.662907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.668272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.700359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fc5d1-5624-48a0-b755-2bc868a52736 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.748504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.749825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.750356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.751447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.752616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:29.753126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.755955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.772830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 386/748] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:29.774497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.782556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.787407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb0bf6-156c-41e0-8e6d-13bad812ec35 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.833219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.834698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.835450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.835926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.837118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:29.838082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.840264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.862344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 363/749] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:29.863482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.895208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9df40-c742-4557-a1ea-b703a95217e5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:29.935388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:29.935938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:29.936213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:29.936481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:29.938358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:29.948369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:29.950113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:29.964980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:29.966189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:29.967298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:29.967298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:29.967298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:29.967298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 387/750] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:29.967298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:29.977801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c1954-141a-41d1-9a30-f91bba4c39f5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.017553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.017946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.018223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.018493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.019221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=89e32c33e2184f5c93cc526f085c5375) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:30.019528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.020950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.026796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 364/751] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/role_assignments?user.id=89e32c33e2184f5c93cc526f085c5375&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:30.028161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.070388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.078816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5840b-9a5b-4a86-b60f-26af9824eb87 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.118277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.118719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.118995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.119261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.181433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.198383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732a45-1bba-4b5b-a16a-b38688c70039 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.199800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.201350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.244885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.245836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.246291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.246756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.247597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.249684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.277857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 365/752] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:30 2022] POST /identity/v3/services => generated 240 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:30.277857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.375909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 388/753] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 07:46:30.384755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.411926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.463227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6ffd-41aa-4edf-854b-b1055a9493c1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.518961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.519827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.521004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.521284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.521971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:30.522815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.524193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.539582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 366/754] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:30.544438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.544566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.563502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c260cc-5ab1-4440-926e-2f9f274bc361 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.606951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.607602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.609693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.609693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.620959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.622434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:30.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:30.642746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:30.643809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:30.643809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:30.643809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:30.643809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 389/755] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:30.649994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.682110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80dccd-7234-4ffc-aabf-2dab9dde13ad None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.733030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.733596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.733918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.734329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.738016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.744987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.748593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:30.776509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:30.777606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:30.780054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 367/756] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:30.783681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.824258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fb51e-3605-488f-98ad-6bb03a616217 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:30.898175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:30.898175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:30.898175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:30.898175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:30.898924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:30.898924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:30.898924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:30.913166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:30.931043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 390/757] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:30.945118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:30.989448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f965c-1507-4dcb-afe8-f0f45dbd2ba0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.058367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.058983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.059261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.059530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.060519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:31.060821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.062307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.082796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 368/758] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:31.089147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.120398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03156-7b5a-4f18-8f11-824d79e0a063 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.167031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.169502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.169806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.170085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.183798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:31.184211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.188398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:31.213296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:31.215481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:31.217645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 391/759] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:31.225495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.279468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88444feb-e6ee-4825-a802-4628daa4f11d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.340434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:31.345647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.346479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.346762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.347037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.361755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.363234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1dfec215-849f-45cf-b7e8-d2bad0548a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:31.381889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:31.382898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:31.384219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:31.384219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:31.384219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:31.384219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 369/760] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:31.389859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752ff3-787c-4649-b8a0-76cc41caa778 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.390293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752ff3-787c-4649-b8a0-76cc41caa778 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.390713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752ff3-787c-4649-b8a0-76cc41caa778 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.393004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 370/761] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:31.393763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38752ff3-787c-4649-b8a0-76cc41caa778 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.412261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502037e2-812f-47e7-8eb0-cfee2cef406b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.456972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.458137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.458417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.458815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.459695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:31.460004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.462489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9b676-43c0-468d-be96-56f718b0d109 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.479784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 392/762] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:31.481008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.481292 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.481564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.506033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:31.507949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:31.510180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38752ff3-787c-4649-b8a0-76cc41caa778 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.518829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:31.557994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.558643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.558887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.559342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.560409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:31.560863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.562437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.579384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 371/763] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:31.580348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.601616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 393/764] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:31.602892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.660133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:31.739724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1af6cc-5583-416e-8452-770eb6d7bd68 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.756764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed278de-25ce-4725-afd6-708e1741faf7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:31.798575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.799210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.799483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.799752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.812210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.813570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.815103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:31.815772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:31.816048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:31.816316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:31.817032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=6555dc699d9d434eb866eaf08805ccb6) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:31.817332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:31.818754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:31.829268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 372/765] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/role_assignments?user.id=6555dc699d9d434eb866eaf08805ccb6&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:31.835312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:31.836258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:31.837199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 394/766] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:31.844085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:31.903085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:31.985439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.045944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f2639-8dff-4003-ba40-8d0c88e30583 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.120231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.120773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.121050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.121321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.122053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:32.122359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.123807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.133455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e86-e3ed-49f5-aa98-bc6f4b05d5a6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.151020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 373/767] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:32.152983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:32.158276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.203969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.204528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.204813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.205086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.205672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.207079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:32.225169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:32.226252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:32.227107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:32.235216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 395/768] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 07:46:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:32.236125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.313625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:32.326127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c77ee-83a0-4d75-bc3c-ac321ea8b35c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.364994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.366142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.366653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.367132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.369491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=742f83bd38b542d193a911ef623abd45, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.371315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.389549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 374/769] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:32 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:32.424790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea76d3-f805-4bee-a1cb-0171ae4cd383 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.463608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8628eb8-9042-444c-acec-3570815433af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.464239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628eb8-9042-444c-acec-3570815433af None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.464515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628eb8-9042-444c-acec-3570815433af None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.464787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8628eb8-9042-444c-acec-3570815433af None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.465430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628eb8-9042-444c-acec-3570815433af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:32.465758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628eb8-9042-444c-acec-3570815433af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.467168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8628eb8-9042-444c-acec-3570815433af None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.472804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 396/770] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:32.478175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.487518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ce617-3f11-486f-a05d-62b1e27a7bfa None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.488119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ce617-3f11-486f-a05d-62b1e27a7bfa None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.488615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ce617-3f11-486f-a05d-62b1e27a7bfa None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.491280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 397/771] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:32.502109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.502634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.503057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.545297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:32.547038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:32.619906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:32.663036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 398/772] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:32.668729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ef841-7969-417a-be33-f9df3e01165e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.704982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.705601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.705922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.706197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.706835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:32.707137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.708585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.714813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 375/773] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:32.720266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.823155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbed79-f99c-4227-a8a4-a552ce9218bf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:32.861362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:32.862031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.862308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.862578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.863301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] RBAC: Adding query filter params (interface=public, service_id=d08518a68ee24d0cba80267cc2110e5b, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:32.863609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:32.865028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f789cae-3296-44ba-8a32-e05742bfe42d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:32.875428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 399/774] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:46:32 2022] GET /identity/v3/endpoints?service_id=d08518a68ee24d0cba80267cc2110e5b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:32.883224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:32.923549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:32.971039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747c9b-7efc-468e-afa6-2b63cd9893b5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.971375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747c9b-7efc-468e-afa6-2b63cd9893b5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.971607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747c9b-7efc-468e-afa6-2b63cd9893b5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:32.974751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 400/775] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:32.979014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:32.979299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:32.979574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.017457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:33.018682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:33.061988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:33.065418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a6f0c-2a28-42af-86cd-77f7633a3daa None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:33.102075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:33.102681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:33.102955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:33.103222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.103823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:33.105180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:33.113905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 376/776] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:32 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:33.114814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.120248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 401/777] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:33.210017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:33.234890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9382e-729c-4d74-8907-92d475eb3948 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:33.287225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:33.288444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:33.288722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:33.288991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.306766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:33.311943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:33.312866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:33.313699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 377/778] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:33.319258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.436541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118f361-e07f-4510-8614-4fd02c00302e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:33.472066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:33.472604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:33.472877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:33.473151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.473858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:33.474158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:33.476087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:33.484643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 402/779] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:33.490218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.532332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.571128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:33.659466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e165e-1ba0-4da9-b4b4-0a766d38d038 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:33.707228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a386e0c-0f7a-4a3c-95ba-661b75391882 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:33.721892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:33.722630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:33.722818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:33.723066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.737786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:33.741162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:33.742664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:33.765189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:33.766204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:33.767090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:33.767090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:33.767090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:33.770357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:33.770987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:33.771318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:33.771587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:33.772769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:33.774192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:33.778867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 378/780] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:33.784350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:33.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:33.789390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:33.790277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:33.797190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 403/781] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/services/network => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:33.801811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:33.970384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:33.974963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da39db-a37e-494f-906e-d90c6846206a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.012956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.013703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.014019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.014295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.014949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:34.015251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.016380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984a99-ee09-4a08-a070-2361010a62f8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.016630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.030585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 379/782] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:34.035816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.093477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.094111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.094385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.094651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.095283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] RBAC: Adding query filter params (name=network) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:34.095581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.097057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.102231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 404/783] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:34.107337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.138356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:34.229035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5dea7-3c1d-403d-bf6e-f51047012367 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.272355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fbb85-0acd-4271-a894-c227cda93a26 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.309542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.309774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.310424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.310530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.310707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.310800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.310959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.311050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.311681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] RBAC: Adding query filter params (type=network) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:34.311986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.313350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.324788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 405/784] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:34.325639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.327114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.330306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.353670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1c6f70fd-ed53-425b-882c-adefb2f1943b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:34.355396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:34.356536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:34.357472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:34.360630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 380/785] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:34.367327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab59d0-87ee-47d2-97c3-e34b8fb15111 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.368231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab59d0-87ee-47d2-97c3-e34b8fb15111 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.368665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab59d0-87ee-47d2-97c3-e34b8fb15111 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.370397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 381/786] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:34.371254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab59d0-87ee-47d2-97c3-e34b8fb15111 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.471872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48b1fa-2536-4dd1-9359-0dad44c00756 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.511718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.512407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.512802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.513173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.514120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] RBAC: Adding query filter params (interface=public, service_id=5a07269e73a94b6bb5acbe390c6cef4b, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:34.514639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.516151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6dc8e-7327-45f3-973b-a17441ed9f68 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.526211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 406/787] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/endpoints?service_id=5a07269e73a94b6bb5acbe390c6cef4b&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:34.527429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.527785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.528140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.548351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:34.549811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:34.608110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab59d0-87ee-47d2-97c3-e34b8fb15111 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.623770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:34.638107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 407/788] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:34.639300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.651414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.652080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.652357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.652663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.653400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:34.653702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.657754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:34.657875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.678465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 382/789] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:34.685976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.798914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea3624-327d-4d52-a17f-c71a643df426 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.854235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9075d95-f5e4-4d36-8dee-ad36a21eadb8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:34.856705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.857342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.857614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.857958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.893776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.893776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:34.924572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:34.925585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:34.926536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:34.926536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:34.926536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:34.926536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 383/790] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:34.933585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:34.935751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:34.936360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:34.936634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:34.936904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:34.937513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:34.939329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f3711-6705-4aee-b50b-8ecee73e5155 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:34.953203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 408/791] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/services => generated 772 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:34.970216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.027105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263a3e-dab5-4d96-94d9-2f1d351cb271 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.027743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263a3e-dab5-4d96-94d9-2f1d351cb271 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.028246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263a3e-dab5-4d96-94d9-2f1d351cb271 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.034173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 409/792] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:35.039303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263a3e-dab5-4d96-94d9-2f1d351cb271 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.066820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd02fb-e44d-464f-88d8-d02d701dade5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.104730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.105702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.106199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.106685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.107534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:35.108021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.109584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e18b-4857-4cea-85a0-c8c30ddfdb2d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.122077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 384/793] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:35.123726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.124155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.124564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.143820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:35.145307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:35.152393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.187021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263a3e-dab5-4d96-94d9-2f1d351cb271 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.234311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.234665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 385/794] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:35.235387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.235680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.236133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.236595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.254296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.264589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:35.276131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:35.277086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:35.278141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 410/795] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:35.287335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.436867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf943-afd9-4f16-91c8-fb5266dff9a0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.446453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ffd6-1986-4b05-b8a3-c0d1c72f0074 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.482018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.485917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.486820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.486934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.487213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.488027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:35.488338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.489762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.499036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 411/796] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:35.507758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.527266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.527903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.528174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.528440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.537642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.539804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:35.557597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:35.558582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:35.559473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:35.562859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 386/797] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:35.564516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.626956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba0098-6933-4ca5-877b-f81fe48d2188 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.659607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.667646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.668373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.668656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.668927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.683578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.685539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:35.703441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:35.704424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:35.705313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:35.705313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:35.705313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:35.705313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 412/798] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:35.706015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.716599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa2a0d-3982-42ce-86f8-6dce7582e721 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.755042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.765971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.766982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.767377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.768022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.782920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:35.790388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:35.791498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:35.792354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:35.792999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 387/799] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:35.793887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.822577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab9c78-df2f-4894-8fdc-3f357eed5b0f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.860987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.861760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.862040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.862320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.863138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:35.863445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.866406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf11864-e40b-406c-accf-b9f17326fed6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.885222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 413/800] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:35.886424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6d2a-1342-4a34-954a-91dc5a094872 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.886705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6d2a-1342-4a34-954a-91dc5a094872 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.886982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac6d2a-1342-4a34-954a-91dc5a094872 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.892201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 414/801] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:35.892201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac6d2a-1342-4a34-954a-91dc5a094872 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:35.904224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de0f82-97fd-4c95-abe5-03d2e30e07a9 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:35.934750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:35.943264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:35.944054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:35.944337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:35.944610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:35.945254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:35.945557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:35.946969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:35.960425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 388/802] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:35.961294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.006216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac6d2a-1342-4a34-954a-91dc5a094872 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.013277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.041743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.042360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.042640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.042907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.043635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:36.043856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.045210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68680724-d2ad-4f07-bc34-2002553cd1e1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.058739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 415/803] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:36.058739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.058739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.058739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.074829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:36.075403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6defb32-99c1-4cb4-8cc6-999bd986909a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.076009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:36.110105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.110761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.111039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.111315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/89e32c33e2184f5c93cc526f085c5375/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.113228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=89e32c33e2184f5c93cc526f085c5375, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.114690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.128307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 389/804] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:35 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/89e32c33e2184f5c93cc526f085c5375/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:36.129388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.175888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 416/805] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:36.176458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.207189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.308562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa84d-559e-4b61-bae5-70c9cb96da44 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.332182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.339723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23a3e6-0df5-4cdf-8a5b-bd2658257c92 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.360644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.361602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.362084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.362484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.371612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.373847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.387328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.387968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.388247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:36.388481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:36.389385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:36.390380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:36.390909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.390979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 390/806] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:36.395789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.413442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.414954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.539538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28188f51-60ea-4894-9875-760fe6be5a9d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.558014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.578294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.578831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.579111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.579384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.580125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:36.580434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.581906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.606978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 391/807] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:36.608229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:36.629374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:36.630425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:36.632014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:36.632014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:36.632014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:36.632014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 417/808] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:36.632487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.739657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b6107-bc43-4db5-929e-002c092a900b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.742052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fdac4-0fe7-45b9-a88b-3331f9eeaa6c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.780077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.780806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.781086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.781359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.782034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.784119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.785083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.785985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.786274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.786548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:36.794687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:36.796959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:36.798926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 392/809] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:36.800141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.800225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.800225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:36.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:36.823868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:36.825437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:36.825437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:36.825437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:36.825437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 418/810] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:36.825437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:36.835163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.840881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:36.936809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530058-b7de-42d6-b782-af0326bb17e3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.962934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733812a-aed8-4a34-9b83-e5a732788fa6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:36.986018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:36.986982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:36.987270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:36.987543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:36.988197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:36.988497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:36.991165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.002128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.003038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.003519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.003992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.004847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:37.005348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.007194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 393/811] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.007952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.008130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.021092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 419/812] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.022033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.102407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.116660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.126900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca9451-f399-4860-8643-389c3a11e554 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.145032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f87a-16f7-4c06-b11c-7671697e2b00 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.173298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.173958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.174396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.174667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.175773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:37.176203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.178585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.183723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.184492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.184870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.185489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.193235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.195392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.197883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 394/813] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.198841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:37.214632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:37.215742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 420/814] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:37.217056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.319344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd024a-b131-4159-af9c-e0c4199f0fe7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.337306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da0cd6-6434-4733-8dfd-be93ca2772be None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.354850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.355395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.355670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.356944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:37.357126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.358499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.366159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 395/815] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.366333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.366421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.376044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.376680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.376956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.377223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.386323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.387812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.389090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7801e9c1-c89b-44c7-8349-b4a5f2111814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:37.399079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:37.400780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:37.402142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:37.405785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 421/816] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:37.407546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20c03-2453-4d07-b9b7-72e199b62c0c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.408040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20c03-2453-4d07-b9b7-72e199b62c0c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.408548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d20c03-2453-4d07-b9b7-72e199b62c0c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.413789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 422/817] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:37.413789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d20c03-2453-4d07-b9b7-72e199b62c0c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.482394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0132ac-0c1a-40c4-bb7b-652e1cc377a7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.515918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.516535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.516805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.517076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.517820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:37.518122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.519534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.526247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d20c03-2453-4d07-b9b7-72e199b62c0c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.536866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 396/818] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.538371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.571544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.572190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.572473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.572748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.573417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.574824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286b73e-97b0-4b93-b455-3f41b2d6c01e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.610003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 423/819] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:46:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:37.611644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.612093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.612507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.624380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.641547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:37.643001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:37.664353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70372d-ad9d-4711-ae1a-76d5ca8c172a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.686737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.728396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.729026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.729305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.729578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.731598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:37.731913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.733390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.738864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 424/820] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:37.740158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.748291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 397/821] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:37.753660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:37.861057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69fc8d-2495-4edf-b101-2dbd51bedfdc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.885581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e9381-9ac8-472a-93db-724e84ba3cc4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:37.908517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.909480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.910040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.910515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.922423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.931224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:37.931952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:37.932231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:37.932507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/6555dc699d9d434eb866eaf08805ccb6/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:37.952742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=6555dc699d9d434eb866eaf08805ccb6, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:37.954310 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:37.963150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:37.978797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 398/822] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:37 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/6555dc699d9d434eb866eaf08805ccb6/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:38.002961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.004449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.067880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.128243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 425/823] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:46:38.134148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.208929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b00b9-fd34-4f67-a11f-01a9d84971b5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.209133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:38.243094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a6e5-faad-4c1c-a1f6-246ce4a6e81f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.258459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.259456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.259941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.260416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] PATH_INFO: `/v3/services/network` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.261210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.262834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:38.273146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:38.274212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 07:46:38.275064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:38.277851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 399/824] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:38.284223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.286585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.287296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.287725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.288059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.299659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.301253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:38.317293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:38.318314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:38.319758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:38.319758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:38.319758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:38.319758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 426/825] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:38.325523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.341608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:38.405360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13ef29-559e-43ad-acb3-e31b73132b43 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.446057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.447138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.447678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.448163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.448555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eaf3c-8a8c-4a01-9f9b-b422e391191a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.449273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] RBAC: Adding query filter params (name=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:38.449827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.451454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.457339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 400/826] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:38.477780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.489027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.489640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.489948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.490220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.491403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:38.491700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.493070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.502036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:38.505938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 427/827] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:38.513641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.640991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd56f-494e-4db7-9d4a-d6fda6521d3c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.651597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c55b4-ec87-4316-87eb-6421a0f190f4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.653340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:38.683324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.683978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.684306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.684735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.686074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:38.686640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.688200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db6625-f5f4-49aa-aa52-c14a960a78b2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.694718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.695433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.695711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.696112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.699481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 401/828] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:38.701095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641485db-9987-4981-92fa-115e9a0ca07e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.701372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641485db-9987-4981-92fa-115e9a0ca07e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.701639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641485db-9987-4981-92fa-115e9a0ca07e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.703164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 402/829] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:38.703846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641485db-9987-4981-92fa-115e9a0ca07e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.710297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.711768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-adf12132-936f-45a3-84b8-72932e24eb1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:38.727066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:38.728013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:38.729003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:38.733554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 428/830] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:38.735473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.735763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.736494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.758502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:38.759680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:38.809665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:38.824711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641485db-9987-4981-92fa-115e9a0ca07e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.840961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 429/831] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:38.841996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:38.899748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:38.900376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:38.900657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:38.900926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:38.901601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:38.908513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:38.955394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0e8b8-ab0f-4a42-9758-bd2693754d4a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:38.965948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 403/832] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:46:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:39.005686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.006436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.006660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.006928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.007754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:39.008057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:39.009548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:39.024471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 430/833] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:39.030550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:39.062994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:39.155297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6feeab-1f6c-48c9-8d38-e9136839d6f6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:39.204029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.204658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.204931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.205200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.206017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:39.206266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:39.207800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:39.218965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 404/834] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:39.238648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:39.274506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:39.359314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f9eef-aeb2-423f-bde1-a060af048d58 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:39.399434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.400087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.400362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.400629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.412321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:39.417726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:39.419887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:46:39.421925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:39.425787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 431/835] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:39.425978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:39.524009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:39.534550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b5b44-fd44-47dd-a164-7a8d9176e949 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:39.572606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.573218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.573495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.573786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.574461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:39.574763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:39.576128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:39.589901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 405/836] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:39.591379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:39.695604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:39.705370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac0bdb-b72d-4cd6-9f1c-a5ced1ca545b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:39.741249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.741906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.742183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.742444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.752827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:39.754295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:39.769316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:39.770300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:39.771216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:39.771216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:39.771216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:39.771216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 432/837] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:39.776910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:39.885030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253f3a-375a-44c5-a81a-528ad83f522f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:39.920359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:39.920977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:39.921259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:39.921532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:39.922211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:39.922516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:39.923892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:39.938142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 406/838] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:39.943650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.048320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd60d5-c5fe-4f1c-82f3-243bdb36354d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.083650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.084307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.084581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.084846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.092809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.094275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:40.105470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:40.106463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:40.107416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 433/839] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:46:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:40.113006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.130152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.220147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162b3b5-2d6f-4eb4-8c76-9bda68fdbb05 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.259818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.260417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.260690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.260955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.261674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:40.262003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.263415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82baf2e-f406-4ee4-b52c-d5cb7beeca49 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.281591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 407/840] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:40.287083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.290662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.352543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37888d-7414-457b-b7d8-28034060be5f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.352718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37888d-7414-457b-b7d8-28034060be5f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.353003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37888d-7414-457b-b7d8-28034060be5f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.354628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 408/841] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:40.359336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b240-0de2-42e7-ab85-0d9e9f8ed6fc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.359620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b240-0de2-42e7-ab85-0d9e9f8ed6fc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.359888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b240-0de2-42e7-ab85-0d9e9f8ed6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.378393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a0d8b240-0de2-42e7-ab85-0d9e9f8ed6fc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:40.379918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a0d8b240-0de2-42e7-ab85-0d9e9f8ed6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:40.401886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252d4d1-c136-419e-8b16-5a71334a69ef None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.437465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.438406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.438880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.439344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.465819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 409/842] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:40.521001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.522944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.524931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.597470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476337-1cd4-4de6-8e07-d5dcc507cc04 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.597847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476337-1cd4-4de6-8e07-d5dcc507cc04 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.598096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4476337-1cd4-4de6-8e07-d5dcc507cc04 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.600321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 410/843] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:40.605304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.605530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.605862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.614203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.628490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:40.630295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:40.632835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 434/844] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:46:40.638209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.703428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 411/845] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:40.745472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a7ad-083a-4ce1-b1a4-dd5f658fbb36 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.746979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.765988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.781621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.782395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.782677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.782943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.794005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.795538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:40.811441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:40.812436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:40.813333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:40.813333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:40.813333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:40.817324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 435/846] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:40.818797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.871309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27493a-bae0-48a7-b690-6db2a0382de8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.902402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:40.907354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.907991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.908265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.908537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.909346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.910921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:40.921065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:40.922303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:40.923499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 412/847] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:40.930621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:40.942167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1d137-bbc4-41b0-ad51-626cff80a7fb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:40.980109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:40.980718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:40.980999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:40.981274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:40.981941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:40.982245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:40.983607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:40.995449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 436/848] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:40.996153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.013079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.050439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f19bc1-58f7-4e29-b91c-0626cd734156 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.085604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.086239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.086519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.086967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.087587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.087888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.089255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.098097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 413/849] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:41.098097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.108106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875729-2397-4071-9c60-301afe2a6b77 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.146019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.146628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.147537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.147809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.157201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.160053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:41.166879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:41.167808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:41.168690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 437/850] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:41.169402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.206113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ad0f-4c88-49f3-8844-cb882a747d07 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.240890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.241524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.241828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.242099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.249052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.250518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.258564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:41.262400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:41.263518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 414/851] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:41.269407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.297290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb307b40-3a45-4506-a813-b3fa267d40e4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.334667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.335495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.335919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.336332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.337124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.337585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.338969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.353764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 438/852] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:41.353764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.381676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061aa67e-da70-4525-86fe-b61337311930 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.409216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.429351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.430352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.431121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.431470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.432577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.432951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.435769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.448361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 415/853] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:41.454819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.481389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6a44-d5cf-4990-9bcb-79b830a39eba None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.530966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.532218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.532591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.532937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.533758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.534178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.536578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.546843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.557494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 439/854] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:41.563685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.604081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c7f8-54d5-4964-ac8d-0aac6a7958cc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.654505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.655146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.655450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.656112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.667973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.669349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.679781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8e66e-791c-4a6d-b8fc-a3185ff2ea90 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-9ee9e398-6e53-493c-b48a-835709daf84d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:41.685126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:41.686278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:41.687207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:41.687207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:41.687207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:41.687993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 416/855] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:41.689299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ede03-b0d3-4120-985a-f644dd8be5a4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.689675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ede03-b0d3-4120-985a-f644dd8be5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.690072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ede03-b0d3-4120-985a-f644dd8be5a4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.694255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 417/856] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:41.695028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ede03-b0d3-4120-985a-f644dd8be5a4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:41.724190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.725455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.725934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.726204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.726983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=89e32c33e2184f5c93cc526f085c5375) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.727325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.728742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d92a5-8591-4365-b2c2-607c0196a751 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.733158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.742129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 440/857] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/role_assignments?user.id=89e32c33e2184f5c93cc526f085c5375&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:41.743390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.743674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.743945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.766401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:41.767663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:41.835187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:41.838407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ede03-b0d3-4120-985a-f644dd8be5a4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:41.848413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 441/858] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:41.887938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:41.888835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:41.889325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:41.889761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:41.890646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:41.891119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:41.892686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:41.917629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 418/859] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:41.919559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.025978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fe19d-9d76-4a15-a784-1f4badba1fc2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.061701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.062595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.063021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.063494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.071773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.073408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:42.086059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:42.087246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:42.088400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:42.093832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 442/860] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:42.096584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.115029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.173301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.215472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.222965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34194a8-ebce-42a9-ae19-d78fef351238 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.232578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8c9d-4583-4d62-a250-0ea1da7e7c91 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.261029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.261851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.262199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.262560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.263438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:42.263866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.265402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.271287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.272132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.272573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.272988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.273846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.275618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.284813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 419/861] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:42.291391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:42.292316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:42.298165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 443/862] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:42.298972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.299045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.413896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a060b9-67ba-44ab-98b4-13e4acfd70ad None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.416808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e351fc-3bd9-4ed5-bb05-b217ada2eebc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.430819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.449905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.450509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.450804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.451054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.451681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:42.451979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.453346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.458781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.459681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.460155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.460869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.463075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 444/863] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:42.464249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.485534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.552686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.555014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.573925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a5ac9-ac58-4079-8cc5-8e202a5da85f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.616956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.617671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.617976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.618345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.618987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:42.619289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.620659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.635029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 445/864] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:42.685095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 420/865] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 07:46:42.690614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.743340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.804912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99c8cb-006d-4ec8-b3f1-cb575617ede5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:42.842754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:42.843392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:42.843672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:42.843939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:42.855894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:42.857364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:42.872464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:42.873436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:42.874810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:42.874810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:42.874810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:42.874810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 446/866] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:42.880730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:42.940567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:42.991252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de5e37-3bd5-43ee-a5c3-c68eff13a63e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:43.027084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:43.028129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.028624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.029140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.030017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:43.030526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:43.032165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:43.048969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 421/867] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:43.051603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:43.153175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:43.160189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d45a3-97ac-44f5-8025-c81b5620304f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:43.196165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:43.197152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.197638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.198152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.205309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:43.207010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:43.219188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:43.220085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:43.221011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:43.228522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 447/868] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:43.230131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:43.344668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef017064-b753-412f-b44a-e900e44ac7f0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:43.358778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:43.382177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:43.383135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.383637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.384120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.385011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:43.385518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:43.387198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335808-3db4-41fb-89ad-0e810baf8686 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:43.405879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 422/869] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:43.407555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:43.513271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b8da-1649-4bcb-a01f-49295274608d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:43.548540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:43.549627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.550164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.550638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.551623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=6555dc699d9d434eb866eaf08805ccb6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:43.552133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:43.553885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4862b-2f20-4bdf-8c42-f2ea998dc399 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:43.565897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 448/870] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:43 2022] GET /identity/v3/role_assignments?user.id=6555dc699d9d434eb866eaf08805ccb6&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:43.702713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1f0f-8ee8-4d4a-8b5e-b845edbab1b3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.703039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1f0f-8ee8-4d4a-8b5e-b845edbab1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.703264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1f0f-8ee8-4d4a-8b5e-b845edbab1b3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.704938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 423/871] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:43.710689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a12e36-cece-4a78-8858-345aa2d60373 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.710975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a12e36-cece-4a78-8858-345aa2d60373 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.711252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a12e36-cece-4a78-8858-345aa2d60373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.732059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e4a12e36-cece-4a78-8858-345aa2d60373 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:43.733277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e4a12e36-cece-4a78-8858-345aa2d60373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:43.753542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:43.809092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 449/872] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:43.985634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f375ca-925c-44d3-8a50-a10a14c3d167 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.985941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f375ca-925c-44d3-8a50-a10a14c3d167 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.986223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f375ca-925c-44d3-8a50-a10a14c3d167 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:43.987841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 424/873] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:43.995737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:43.996140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:43.996517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.015952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:44.017133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:44.080913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f375ca-925c-44d3-8a50-a10a14c3d167 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:44.088529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 450/874] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:44.198926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f375ca-925c-44d3-8a50-a10a14c3d167 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:44.233610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:44.234235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.234512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.234781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.235443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:44.236805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:44.256092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 425/875] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:44 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:44.368381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:44.457633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:44.482295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2640ff-8252-42f4-9b6e-a6fe2811d502 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:44.524388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:44.525025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.525298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.525566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.526184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:44.527566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:44.536762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:44.538362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:44.539191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 451/876] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:44.545139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:44.577711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:44.652826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5167b-3dda-40d2-8df1-cc58de7df00a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:44.688558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:44.689520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.690052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.690525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.691366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:44.691872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:44.693478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea36fb-91f4-4e1a-a775-06c826b58ae2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:44.700172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 426/877] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:44.705955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:44.823572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376fe99-5c71-47e8-9bcc-5dd1a6bc31da None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:44.859567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:44.860543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.861118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.861593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.862444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:44.862953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:44.864514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c467-9839-4434-8859-2dc9b912796a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:44.869987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 452/878] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:44.913804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e03b8-7182-415a-840e-87ddbee79bc6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.914421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e03b8-7182-415a-840e-87ddbee79bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.914898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e03b8-7182-415a-840e-87ddbee79bc6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.916747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 427/879] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:44.922136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04f8a-4b99-4474-8e25-6ec9666cd815 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:44.922754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04f8a-4b99-4474-8e25-6ec9666cd815 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:44.923233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a04f8a-4b99-4474-8e25-6ec9666cd815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:44.931357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:44.945251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-31a04f8a-4b99-4474-8e25-6ec9666cd815 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:44.947554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-31a04f8a-4b99-4474-8e25-6ec9666cd815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:45.025341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 453/880] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:45.301906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e03b8-7182-415a-840e-87ddbee79bc6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:45.399955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:45.412134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e03b8-7182-415a-840e-87ddbee79bc6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:45.449179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:45.450222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:45.450705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:45.451175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:45.452030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:45.453616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9087ebb-c1ed-4167-be4e-f6c672f41db1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:45.471214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 428/881] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:45 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:46.046858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a4a47-b100-4061-b8e5-2f18a7006945 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.047883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a4a47-b100-4061-b8e5-2f18a7006945 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.048390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a4a47-b100-4061-b8e5-2f18a7006945 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.050299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 454/882] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:46.055670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.056205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.056689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.075707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:46.077231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:46.152587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 429/883] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:46.429283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a4a47-b100-4061-b8e5-2f18a7006945 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:46.543472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a4a47-b100-4061-b8e5-2f18a7006945 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:46.582612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:46.583690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.584731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.585209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.587322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:46.597301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9a65c771-1b98-47a0-81f4-5c8a8473c9e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:46.603144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:46.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:46.605099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:46.607784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 455/884] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:46:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:46.613902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:46.621529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221b26-8bea-40ae-a3c1-7fe29f432538 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.621841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221b26-8bea-40ae-a3c1-7fe29f432538 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.622121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7221b26-8bea-40ae-a3c1-7fe29f432538 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.623820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 456/885] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:46.629099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.629382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.629659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.650204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:46.651508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:46.720820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:46.736163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 457/886] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:46.748166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987240b-c7c2-4e38-9d53-126bf68e79dd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:46.784195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:46.784851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.785131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.785447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.786144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:46.786442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:46.787880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:46.796806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 430/887] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:46:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:46.803151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:46.910071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadab6e4-0221-4712-a9e3-9db222b02ab1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:46.938936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:46.951559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b53-031c-4926-980d-661403845c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:46.952652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b53-031c-4926-980d-661403845c44 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:46.953128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b53-031c-4926-980d-661403845c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:46.953605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2144b53-031c-4926-980d-661403845c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:46.966657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2144b53-031c-4926-980d-661403845c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:46.968464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2144b53-031c-4926-980d-661403845c44 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f2144b53-031c-4926-980d-661403845c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:46.984906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:46.985976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:46.986889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:46.986889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:46.986889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:46.988829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 458/888] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:46.994603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.030481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b53-031c-4926-980d-661403845c44 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.100446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388b3a-a5bc-4d01-b312-e5df2e380e07 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.135384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.136007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.136290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.136565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.137213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:47.137518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.138704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2144b53-031c-4926-980d-661403845c44 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.139167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2163a-b1f9-4561-93df-bdd1345b6f4d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.151934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 431/889] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:47.153156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ba95e-9246-4362-aa90-5f6241ac3319 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.153437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ba95e-9246-4362-aa90-5f6241ac3319 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.153708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ba95e-9246-4362-aa90-5f6241ac3319 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.157864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 432/890] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:47.159873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ba95e-9246-4362-aa90-5f6241ac3319 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.164352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.181892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.182791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.183218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.183696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.184283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.186003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-dc79ca38-2961-41b9-90d5-340f1a057f75 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:47.195809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:47.196826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:47.197690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 459/891] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:47.199217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.199217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.199542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.218481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:47.219674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:47.260391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.276608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ba95e-9246-4362-aa90-5f6241ac3319 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.299440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 460/892] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:47.300674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.330804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.331497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.331777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.332047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.340244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.342037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:47.355071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:47.356065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:47.357269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 433/893] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:47.369843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.427835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec2d41-49d2-4fdc-aafa-927a0e6b7c41 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.462441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.471653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.473019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.473522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.474279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.475959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:47.476458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.478771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.485358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 461/894] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:47.491254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.501066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ab84-2693-49de-b4c6-d80ca0f2b743 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.534601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.544295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.545417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.545900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.546239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.547102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:47.547482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.549223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.570418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 434/895] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:47.576502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.612882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a233fe-6d45-490a-a575-d4aa175eb45e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.648856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.649478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.649908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.650184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.650935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:47.651401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.652775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.657923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 462/896] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:47.658741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.684267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf9aad-95cf-42d8-bd8f-0d057aba033f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.706686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.719120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.719731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.720004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.720271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.756454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.760371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045f480-04d4-4f85-b004-78a794161ec3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.789207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.790610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0e87d-8c58-4ffb-a435-9c6bcf809237 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.795536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.796138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.796413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.796686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.797329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.798713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed287027-56fc-4783-be26-449cdaaafb32 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:47.813101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 463/897] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:47 2022] POST /identity/v3/services => generated 241 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:47.814338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f55769-3744-49cc-b593-dedeb288a497 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.814619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f55769-3744-49cc-b593-dedeb288a497 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.814893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f55769-3744-49cc-b593-dedeb288a497 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.816105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 464/898] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:47.816818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f55769-3744-49cc-b593-dedeb288a497 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:47.918995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 435/899] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:46:47.920351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.920639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.920913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.940902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:47.942194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:47.953946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f55769-3744-49cc-b593-dedeb288a497 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:47.963183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:47.993223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:47.993872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:47.994157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:47.994432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:47.995144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] RBAC: Adding query filter params (interface=public, service_id=45edfba58ecf49698dfdda37ca9e51d2, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:47.995456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:47.996838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.002745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 465/900] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:46:47 2022] GET /identity/v3/endpoints?service_id=45edfba58ecf49698dfdda37ca9e51d2&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:48.003626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.023461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 436/901] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:48.024541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.025607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.139769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d48906-fef0-43d9-bb7a-ab4fe68c4497 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.156316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41815b-f9bb-47a6-99d7-caec9dacebbd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.173871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.174510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.174784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.175054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.186734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.188179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.200852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.201795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.202293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.202778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:48.203463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:48.204386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:48.205268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:48.205268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:48.205268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:48.206016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.207609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.209286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 466/902] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:48.210308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.216384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 437/903] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:46:48.293935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.322731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.331054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517ed7-3f28-4c6c-be70-0dc656b17227 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.346149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.376153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.377279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.377570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.377897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.379004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:48.379312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.382195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.399150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 467/904] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:48.404308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.461449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee389cc-bbc1-4a1a-ad7c-eb59b8ea742f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.507516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.508152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.508425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.509105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.509697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.511521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.521066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbb9f0-2c53-4987-ac5c-d04f7c35d203 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:48.522775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:48.523687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:48.524569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:48.525295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 438/905] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:48.533128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.558818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.566907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.567549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.567827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.568132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.576255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.577710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:48.590499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:48.591650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:48.592667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:48.596515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 468/906] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:48.597718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.649466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.676022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c60aa-ad1d-4604-8995-3ad82f18aadc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.715896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42de21e-26c1-45d4-bf71-ca150658f0a0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.730145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.731323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.731729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.732162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.732928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:48.733354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.734997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.745697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 439/907] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:48.746899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.758634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.759077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.759359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.759633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.760333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:48.760645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.762092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.774953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 469/908] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:48.781045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.854350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.859472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3478a94-ef35-44a7-be39-69671f52a1bd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.904376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.905029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.905306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.905628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.906301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:48.906599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.907966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.918330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 440/909] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:48.919968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:48.930125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf51fa-ad64-41f6-b79d-0b50ecc1456e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:48.935099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:48.966904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:48.967666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:48.967945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:48.968216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:48.968935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] RBAC: Adding query filter params (role.id=bc83cec3ef0247c2911858049a3f3e4c, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=6555dc699d9d434eb866eaf08805ccb6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:48.969300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:48.970882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:48.983035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 470/910] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/role_assignments?user.id=6555dc699d9d434eb866eaf08805ccb6&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=bc83cec3ef0247c2911858049a3f3e4c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:49.038351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7fb58-2fba-4435-964f-ef4616018ea0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:49.076580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:49.077583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:49.078115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:49.078615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:49.079567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] RBAC: Adding query filter params (interface=public, service_id=834347ca42e24579a576534d12c82b6b, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:49.080093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:49.081689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f59663-fd9b-4e96-8f7b-fa6f57b7ad67 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:49.091269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 441/911] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:46:48 2022] GET /identity/v3/endpoints?service_id=834347ca42e24579a576534d12c82b6b&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:49.097342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:49.208275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781304-d871-4575-8fa0-f1930f4e26b5 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:49.223693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:49.245944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:49.246933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:49.247420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:49.247884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:49.248671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:49.250293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef218ef2-c6d8-4e54-9101-216577042dc4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:49.256732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 471/912] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:49 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:46:50.076496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d2372-8a36-43de-b185-7ef676d25e9c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.077017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d2372-8a36-43de-b185-7ef676d25e9c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.077017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d2372-8a36-43de-b185-7ef676d25e9c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.078638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 442/913] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:50.084607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe7689-0c37-43c8-aab1-659049d8a4df None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.084888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe7689-0c37-43c8-aab1-659049d8a4df None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.085161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe7689-0c37-43c8-aab1-659049d8a4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.103542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e2fe7689-0c37-43c8-aab1-659049d8a4df None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:50.104720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e2fe7689-0c37-43c8-aab1-659049d8a4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:50.175536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 472/914] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:50.442306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d2372-8a36-43de-b185-7ef676d25e9c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:50.498994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449491-8c09-4a8a-97d9-20cea05a2a9e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.499534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449491-8c09-4a8a-97d9-20cea05a2a9e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.499534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7449491-8c09-4a8a-97d9-20cea05a2a9e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.501134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 473/915] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:50.506998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d284-299c-4571-ae28-7397661910ca None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.507280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d284-299c-4571-ae28-7397661910ca None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.507555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d284-299c-4571-ae28-7397661910ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.516769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:50.529240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3598d284-299c-4571-ae28-7397661910ca None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:50.530577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3598d284-299c-4571-ae28-7397661910ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:50.553509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d2372-8a36-43de-b185-7ef676d25e9c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:50.591822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:50.592458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.592729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.592993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.593568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:50.595084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:50.609757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:50.611034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:50.611886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 443/917] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:50.612702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 474/917] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:50.612782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:50.729242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b5a6-a981-4c39-8898-8cc5d05471bb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:50.770024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:50.770965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.771505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.771990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.772839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:50.773353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:50.775015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:50.777251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:50.782178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 444/918] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:50.791239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d284-299c-4571-ae28-7397661910ca None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:50.829321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:50.890855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d284-299c-4571-ae28-7397661910ca None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:50.913108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:50.926202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:50.926816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:50.927094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:50.927431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:50.928563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:50.928864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:50.930272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:50.935643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 475/919] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:50.943235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.032247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689aca-9fcb-4c7e-9007-7eb07721729a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.052914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1bcfa-5710-4569-ba13-7e382c0bc0dc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.054849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:51.074553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.075462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.075890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.076311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.077481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.079977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.092258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:51.092597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:51.093596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 07:46:51.094480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:51.095239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.095239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.095239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.095239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] RBAC: Adding query filter params (interface=public, service_id=1dc0fdac50f34da8a823f1d14e796621, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:51.095239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.096683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.103595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 445/920] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:51.105436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.109812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 476/921] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:46:50 2022] GET /identity/v3/endpoints?service_id=1dc0fdac50f34da8a823f1d14e796621&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:51.112802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.192224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:51.241932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d45382-57ae-4a74-a28f-a0039c3c2693 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.252114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d941e6-ea12-4751-b063-39292ef1256f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.282570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.283109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.283388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.283661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.284288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:51.284599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.286023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3be779-d91f-4f89-af27-cda2da8406d4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.295997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.296281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 446/922] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:51.297038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.297512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.297617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8ec2-0e87-426e-a51c-43e374f04028 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.297857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8ec2-0e87-426e-a51c-43e374f04028 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.298130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8ec2-0e87-426e-a51c-43e374f04028 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.298455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.299232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.299335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 447/923] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:51.301235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edad45-0598-4bf5-ac0c-5442e01e5ffa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.302861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8ec2-0e87-426e-a51c-43e374f04028 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.317770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 477/924] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:46:51.317770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee1012-c1bc-4eb6-a3ba-488d968c2e71 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.317770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee1012-c1bc-4eb6-a3ba-488d968c2e71 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.317770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee1012-c1bc-4eb6-a3ba-488d968c2e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.335743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7eee1012-c1bc-4eb6-a3ba-488d968c2e71 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:51.336968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7eee1012-c1bc-4eb6-a3ba-488d968c2e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:51.365537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:51.425762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 478/925] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:51.444712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8ec2-0e87-426e-a51c-43e374f04028 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.475071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:51.484912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.485921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.486420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.486955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.487796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:51.488314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.490009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.493354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8496b-b2cc-4512-9492-93b3535d37b6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.493951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8496b-b2cc-4512-9492-93b3535d37b6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.494503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8496b-b2cc-4512-9492-93b3535d37b6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.496406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 479/926] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:51.498338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 448/927] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:51.502418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.502902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.502991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.504336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.529297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:51.531321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:51.630140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 480/928] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:51.637881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237d5a9-58fd-41dc-b97f-2a625c99ab9b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.666163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.666770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.667044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.667313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.667977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:51.669332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:51.692645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 449/929] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:46:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:46:51.705664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.818450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:51.829566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a1bff-6c93-4d6c-a236-02b943d80242 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:51.874092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:51.874741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:51.875013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:51.875278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:51.893107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:51.898561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:51.899482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:51.900324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 481/930] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:51.907470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.952462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:51.970083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.027943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244415a2-a121-4ba2-b538-6ace9195d3a8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.058887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea99463-3e50-4586-a8a2-4306897201d0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.074610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.075937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.076640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.077063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.078087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:52.078626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.079787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.092350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 450/931] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:52.098958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.104368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.105064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.105351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.105623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.106296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.107695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:52.117770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:52.118737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:52.119631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 482/932] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 07:46:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:52.124929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.138746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.222844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f24eb3-6272-4701-b8a7-a418bff9d0cc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.234743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.244434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88588160-c994-4f45-b191-7bb2da5f7724 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.264931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.265624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.265950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.266224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.278139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.279639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.286368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.288069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.288562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.289036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.289945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:52.290453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.292042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.297840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 483/933] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:52.298062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:52.298959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:52.305373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:52.305373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:52.305373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:52.305373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 451/934] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:52.305373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.306665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.391140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.416841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34eb8b-fa92-44e2-9d7a-d3fe0f757d2a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.419020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fa18a-c5aa-4143-92d6-8bc98acf9dee None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.454014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.455125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.455694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.456201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.457214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:52.457771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.458195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.458920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.459251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.459596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.460260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.460367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:52.460661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.462110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.468088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 484/935] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:52.473194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.478182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 452/936] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:52.483601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.511785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.594870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf70a8d-0c72-4a7e-b3ed-aa19c0cd9895 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.614557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49b79-dcf4-4802-9782-9602cc109cec None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.648135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.649261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.649838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.650350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.651504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.652111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.652387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.652655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.653448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.654937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.659996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.662974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.664125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:52.678606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:52.679628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:52.680580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:52.688016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 485/938] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:46:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:46:52.688396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 453/938] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:52.689244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.814069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2140af-732a-4bea-96ea-83ad20027813 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:52.851851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:52.852785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:52.853308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:52.853815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:52.854737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:52.855222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:52.856848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:52.875950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 486/939] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:52.882522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:52.974819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:52.995207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492b0fc-e376-45eb-9ec2-85dd433373ca None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:53.031181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:53.031819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.032103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.032372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.044158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:53.045598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:53.060931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:53.061946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:53.062846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:53.062846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:53.062846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:53.062846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 454/940] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:53.068145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:53.119328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:53.176088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3688e-63a2-47be-b0be-6c44e118f86b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:53.212179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:53.213197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.213741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.214239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.215115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:53.215619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:53.217195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:53.230411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 487/941] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:53.235632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:53.343342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea6480-114e-42b3-9495-15c2eb38577d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:53.382187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:53.382820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.383095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.383365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.385260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:53.390504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:53.391948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:53.404509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:53.405447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:53.406398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 455/942] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:53.412175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:53.526753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17de8f-f86e-4de3-921b-86d93ebef98a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:53.536432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:53.563694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:53.564331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.564603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.564885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.565610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:53.565970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:53.567496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a417e-0313-4e7b-a160-a183485447c8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:53.580713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 488/943] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:53.587023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:53.601140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44362be-18ac-4d66-814d-b8338db20a05 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.601440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44362be-18ac-4d66-814d-b8338db20a05 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.602049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44362be-18ac-4d66-814d-b8338db20a05 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.603382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 489/944] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:46:53.609252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.609535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.609852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.631532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:53.632919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:53.705877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402f6b-1376-4de2-839b-bc6153032468 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:53.722112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 490/945] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:53.745960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:53.747009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:53.747498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:53.748038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/6555dc699d9d434eb866eaf08805ccb6/roles/bc83cec3ef0247c2911858049a3f3e4c` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:53.751122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=6555dc699d9d434eb866eaf08805ccb6, role_id=bc83cec3ef0247c2911858049a3f3e4c)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:53.754094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecaa61-bd95-4ba9-baba-4646d0fac42b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:53.773523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 456/946] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:46:53 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/6555dc699d9d434eb866eaf08805ccb6/roles/bc83cec3ef0247c2911858049a3f3e4c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:46:53.994075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.064980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a85a90-2d5d-4b73-ace4-4760ff880d0e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.065324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a85a90-2d5d-4b73-ace4-4760ff880d0e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.065567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a85a90-2d5d-4b73-ace4-4760ff880d0e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.067205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 457/947] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:54.072957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.073240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.073516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.091915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:54.093102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:54.099023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:54.103216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b085e-c95e-4741-81aa-b42bed36cc5d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.147893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.148557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.148907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.149397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:54.150274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.150986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.152380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:54.164672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:54.165725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:54.166786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 491/948] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:54.172614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.174197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 458/949] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:54.279693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b6d1-eb9a-4e2a-8c03-4128bf08f9bb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.314819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.315431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.315718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.315990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.316688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:54.316989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.319133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.324301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 492/950] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:54.329984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.439749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a4c5-08fb-4b92-9412-bbe849c4c40a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.474765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.475383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.475666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.475937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.476581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:54.476885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.478300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.483567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 459/951] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:54.489311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.496670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.525913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:54.554694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:54.615394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21acfe-cb44-4c75-9cac-f2b67ee7e81f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.616387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f1cb3-c00f-44af-b792-100e706865cb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.654978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.655617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.655891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.656163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.656873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.658580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.660873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.662080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.662360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.662629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.663296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.664771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.668613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING keystone.server.flask.common [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 07:46:54.689559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 07:46:54.689559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 07:46:54.689559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [parameters: {'id': '884265ee8cb74409a2c94c46f8b52906', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 07:46:54.689559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 07:46:54.694308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 493/952] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:46:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 884265ee8cb74409a2c94c46f8b52906.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 884265ee8cb74409a2c94c46f8b52906. Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:46:54.698262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 07:46:54.699143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 13 07:46:54.700106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 07:46:54.701025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 07:46:54.702875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:46:54.704733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'id': '884265ee8cb74409a2c94c46f8b52906', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.706170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:54.707314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 884265ee8cb74409a2c94c46f8b52906. Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:54.708487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 460/953] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Sep 13 07:46:54 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:54.709318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:54.812196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:54.842871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bef684-9966-4c78-863d-48e8c3dde94a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:54.892676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:54.893601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:54.900025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:54.900621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:54.908546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:54.910294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-3a1fd1ce-7da3-4f9e-98e9-9600596e9347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:54.922706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:54.923653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:54.924615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:54.929063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 494/954] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:54.933441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:55.044502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8a4c3-2262-43ae-a788-b3b4c8a710b1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:55.047823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:55.088444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:55.089363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.089820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.090237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.091054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:55.091604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:55.093470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d2b5c-3ecd-4458-b557-26a31167f115 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:55.114920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 461/955] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:55.175936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe550-c3dc-41a4-bc40-042e4565f454 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.176507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe550-c3dc-41a4-bc40-042e4565f454 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.176961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe550-c3dc-41a4-bc40-042e4565f454 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.178715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 495/956] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:55.184989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.185271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.185536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.205514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:55.207117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:55.289286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 462/957] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:55.557048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe550-c3dc-41a4-bc40-042e4565f454 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:55.668630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe550-c3dc-41a4-bc40-042e4565f454 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:55.708080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:55.709047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.709509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.710028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.710795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:55.712373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:55.721818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:55.722889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:46:55.723763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:55.726153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 496/958] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:55.733796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:55.828971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:55.840103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1e4e4-cc2d-4ba9-bbb1-033e09bbf76f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:55.881071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:55.881694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.887212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.887505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.888170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:55.888480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:55.890010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e2b97-4609-4bd3-bd80-37065d0e4905 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:55.901086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 463/959] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:55.906486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:55.909932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:55.966675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49cb1-6d30-494f-9726-484e1d3f6037 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.967026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49cb1-6d30-494f-9726-484e1d3f6037 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.967223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc49cb1-6d30-494f-9726-484e1d3f6037 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.968846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 464/960] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:55.975017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:55.975410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:55.975792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:55.995083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:55.996482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:56.019310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459f2c5-e738-443b-9de3-b5a0f0f151c8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:56.059488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:56.060395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:56.060673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:56.061052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:56.061692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:56.062126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:56.063505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:56.069648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 497/961] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:46:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:56.077944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 465/962] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:46:56.346754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:56.418904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:56.460066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa9c3-4a96-4b04-8b09-103bb0fca3a8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:56.496827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:56.497766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:56.498197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:56.498637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:56.509462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:46:56.514470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:56.515607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:46:56.516696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:56.519086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 498/963] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:46:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:56.527676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:56.619802 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:56.632637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff723c0f-f2c1-4e38-8f61-f90973011d2d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:56.668576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:56.669192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:56.669470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:56.669757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:56.670434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:56.670735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:56.672095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:56.685533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 466/964] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:46:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:56.687126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:56.792852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c266ae7-bc01-43c7-a5ac-7570ba96736f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:56.820254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:56.829296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:56.829962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:56.830246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:56.830702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:56.843068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:56.844536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b17647d7-6189-47fd-a6af-1a79a9758933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:56.859822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:56.860847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:56.862143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:56.862143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:56.862143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:56.862143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 499/965] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:56.868404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.004982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1a76-b91e-49e2-baad-175652edb764 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.030651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a19ca-24e2-4873-8b33-77295d5c2cd0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.030959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a19ca-24e2-4873-8b33-77295d5c2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.031199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a19ca-24e2-4873-8b33-77295d5c2cd0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.032775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 500/966] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:57.037722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.038036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.038302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.044627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.045502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.046125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.046404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.047054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:57.047356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.048730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.060011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:57.061518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:57.066454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 467/967] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:57.068236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.080386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.150123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 501/968] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:57.181371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305ee60-0326-4f1b-89a7-774c5ed55a1f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.216817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.217450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.217726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.218070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.225441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.226916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:57.238761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:57.240208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 07:46:57.241154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:57.244943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 468/969] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:57.246566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.350327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfbc89-62ce-4386-89ac-88992f5d8806 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.361591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.391023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.391641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.391918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.392187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.392924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:57.393225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.394690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.417378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 502/970] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:57.418674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.433858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.504516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.536121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd77fd-67fe-41c9-9966-d0507c410bb0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.539295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58dc6e-4d50-4ea7-9d2b-a7e2cb14fc90 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.571588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.572212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.572492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.572765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.573341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.574738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.575213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.575816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.576089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.576355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-506c93cf-96a8-4b41-b608-37d388cd58d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:57.583697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:57.584733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:57.586148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 503/971] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:46:57.586996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be6ad-792b-475a-87db-8266d6f08876 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.586996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be6ad-792b-475a-87db-8266d6f08876 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.586996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767be6ad-792b-475a-87db-8266d6f08876 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.592257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 504/972] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:57.592257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be6ad-792b-475a-87db-8266d6f08876 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.625398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.655058 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.656665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.700911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767be6ad-792b-475a-87db-8266d6f08876 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.729008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.736579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.736915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.737273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.737470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.738274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:57.738573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.740508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b1808-c2ab-4740-9426-3e2451ed37fb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.749887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 505/973] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:57.751464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.751705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.751988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.769134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 469/974] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:46:57.769464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:57.770788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:57.773628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.841385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 506/975] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:57.842559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.871445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:57.900427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91735651-b649-4111-99b5-8d842f465b4e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:57.948538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:57.949149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:57.949426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:57.949695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:57.950359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:57.950699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:57.952122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:57.959981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 470/976] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:57.965816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:57.966478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbfc39-9e6d-46cd-b2b1-feedbb0dea5b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.003543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.004185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.004463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.004890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.011108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.016157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.017626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:46:58.035950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:58.037502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:46:58.038454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:46:58.038454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:46:58.038454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:58.038454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 507/977] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:58.044322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.093637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31759e1-63a6-4e62-9354-72e183d920f4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.133625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.134208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.134486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.134755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.135463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] RBAC: Adding query filter params (interface=admin, service_id=1dc0fdac50f34da8a823f1d14e796621, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:58.135779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.137135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.142897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 471/978] 10.222.0.15 () {56 vars in 1249 bytes} [Tue Sep 13 07:46:57 2022] GET /identity/v3/endpoints?service_id=1dc0fdac50f34da8a823f1d14e796621&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:46:58.143740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.146651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.172057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9116f-d38e-41af-959c-28f465853681 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.211711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.212238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.212526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.212801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.213549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:58.213771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.215148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.228894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 508/979] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:58.229538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.257360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.261276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3019204-5c8d-4dac-81c4-afc29e14525f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.297990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.298599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.298874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.299139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.299773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.301198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.316140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.sql.core [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 07:46:58.316140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 07:46:58.316140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [parameters: {'user_id': 'acf37c18cee8488ca19f532d16a7af37', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 07:46:58.316140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:46:58.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:46:58.324131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application yield resource Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application next(self.gen) Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise self.value Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application session.commit() Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 07:46:58.330502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.session.flush() Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application compat.raise_( Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 07:46:58.331612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 07:46:58.332731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application conn.query(q) Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application result.read() Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 07:46:58.334376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [parameters: {'user_id': 'acf37c18cee8488ca19f532d16a7af37', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:58.335387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 07:46:58.336374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 472/980] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 07:46:58 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:58.338134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.349607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfaeda-65b8-4aa0-96d7-a8f65b60f1fc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.385523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.386156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.386439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.386715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.387321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.388690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d0210-e80b-4147-8de9-be4c6a6f2c5d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.390398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.396375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 509/981] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:46:58.397887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade32d52-b2ab-4568-a208-d7bd0a4b347a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.398178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade32d52-b2ab-4568-a208-d7bd0a4b347a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.398456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade32d52-b2ab-4568-a208-d7bd0a4b347a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.401866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 510/982] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:46:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:46:58.403019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade32d52-b2ab-4568-a208-d7bd0a4b347a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.434491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea94af-249a-4a1c-a688-d20db243c99f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.474651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.475213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.475488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.475762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.483302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.484998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-014b552b-a642-45de-a7d7-643e34180559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:46:58.498691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:58.499679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:46:58.500644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 473/983] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:58.501619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.501619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.501826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.522287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:46:58.523892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:46:58.529227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade32d52-b2ab-4568-a208-d7bd0a4b347a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.539800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.569202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.569921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.570199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.570463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.578518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.579954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:46:58.595727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:46:58.596724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 511/984] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:46:58.597700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.610784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 474/985] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:46:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:46:58.613024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.680949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.725926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116c05-468a-47da-9c8d-e56bc5ef6b46 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.736928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c060b-70de-4819-a768-e8f14ab7fd84 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.763168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.763775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.764048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.764432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.765131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:58.765433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.766990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.779922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.780537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.780819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.781094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.781817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:58.782126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.783866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:58.797486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 512/986] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:58.803891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.809333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 475/987] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:58.894924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:46:58.914405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.943482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b48247-fe25-46a1-85c2-d9ab72be258c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:58.969168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:46:58.987676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:58.988312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:58.988599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:58.988874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:58.990527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] RBAC: Adding query filter params (role.id=bc83cec3ef0247c2911858049a3f3e4c, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=6555dc699d9d434eb866eaf08805ccb6) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:46:58.991722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:58.993206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703dda2e-6315-45ce-8f27-f1a70210c4bb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:59.001580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 513/988] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:46:58 2022] GET /identity/v3/role_assignments?user.id=6555dc699d9d434eb866eaf08805ccb6&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=bc83cec3ef0247c2911858049a3f3e4c => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:46:59.032421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cbbf-597d-470f-9b4e-6440cb4553ae None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:46:59.079020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:46:59.080068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:46:59.080349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:46:59.080623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:46:59.081305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:46:59.082826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac415-111f-4d82-a1e6-c44c49b00af1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:46:59.104253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 476/989] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:46:58 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:00.604219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb363ac-4388-41cc-8c81-e0b4f400340c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:00.604779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb363ac-4388-41cc-8c81-e0b4f400340c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:00.604779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb363ac-4388-41cc-8c81-e0b4f400340c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:00.609777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 514/990] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:00.611823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:00.612266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:00.612356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:00.630833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:00.632018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:00.695404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:00.709915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 477/991] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:00.978837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb363ac-4388-41cc-8c81-e0b4f400340c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.081873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb363ac-4388-41cc-8c81-e0b4f400340c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.117184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.117876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.118153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.118414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.119013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.120522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-b79b6c20-86ae-481f-9a3d-42ce36e15259 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:01.130511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:01.131593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 07:47:01.132442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.134072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 515/992] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 07:47:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:01.138004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.244857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f3f7d-06f4-4aae-9be2-033211cd5962 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.245144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f3f7d-06f4-4aae-9be2-033211cd5962 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.245423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f3f7d-06f4-4aae-9be2-033211cd5962 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.246933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 516/993] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:01.251828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251d986-5db7-4537-a2ba-6154a0bb771f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.252112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251d986-5db7-4537-a2ba-6154a0bb771f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.252388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251d986-5db7-4537-a2ba-6154a0bb771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.255284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a9b08-0764-4f29-b1cc-301c8b906dcf None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.271807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4251d986-5db7-4537-a2ba-6154a0bb771f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:01.272975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4251d986-5db7-4537-a2ba-6154a0bb771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:01.293073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.293683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.293984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.294254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.294941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:01.295244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.296609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.306375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 478/994] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:01.307373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.326237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:01.351025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 517/995] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:01.352264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250952bf-249d-42c9-9ac8-463e1ed97f55 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.352543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250952bf-249d-42c9-9ac8-463e1ed97f55 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.352811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250952bf-249d-42c9-9ac8-463e1ed97f55 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.354191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 518/996] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:01.359171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.359456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.359721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.372342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:01.377789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:01.378963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:01.417449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76dcf-6dfe-4476-aefb-eef160643ef3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.461217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 519/997] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:01.461396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.461396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.461396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.461396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.461698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:01.461698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.461698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.465210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 479/998] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:01.470652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.570253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378f1d2-a0ad-4a44-8f42-b2b793407f50 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.579933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:01.605888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.606587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.606828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.607154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.607765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.609126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.633777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 520/999] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:47:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:01.634006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.725061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.754283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f286b67-bfbf-413a-9b7d-35d400487359 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.783290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:01.802966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.803602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.804468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.804468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.804730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.806085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:01.816342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:01.818001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:01.818877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 480/1000] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:01.823931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.839472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310448d-e37a-4906-8f91-ad340dbe72d0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.875937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.876575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.876853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.877124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.877710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.879189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:01.893472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:01.894494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:01.900210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 521/1001] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:01.901224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:01.935244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:01.936115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d721-98dc-4762-ac22-79f5b2c7e5d7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:01.973021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:01.973935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:01.974368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:01.974809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:01.975640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] RBAC: Adding query filter params (name=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:01.976105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:01.977613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:01.983838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 481/1002] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:01.989097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:02.011477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7e00-0376-49d4-bd7a-4c6270b10886 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:02.041491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:02.049742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:02.050379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:02.050709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:02.050985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:02.051612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:02.052461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:02.053855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:02.061759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 522/1003] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:02.064889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:02.104112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14726-6d5a-4fef-bc78-ebeb821c8b4c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:02.143506 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:02.144120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:02.144521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:02.144796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:02.145461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] RBAC: Adding query filter params (type=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:02.145786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:02.147161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:02.151585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 482/1004] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:02.166267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:02.173666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551e990-b795-4a91-8b9f-2c18ef6dc5e0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:02.213321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:02.213989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:02.214269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:02.214539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:02.215186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:02.215490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:02.216915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:02.222796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 523/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:47:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:02.229208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:02.339689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef2b15-c174-4b64-95f0-facb384ea0de None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:02.350966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:02.375463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:02.376078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:02.376355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:02.376626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:02.377330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] RBAC: Adding query filter params (interface=public, service_id=411095e1734f49bfafdc562acd7eb45c, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:02.377638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:02.379044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac023ae-e318-4c39-a753-0c7c2c0e442a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:02.384476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 483/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:47:02 2022] GET /identity/v3/endpoints?service_id=411095e1734f49bfafdc562acd7eb45c&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:02.390409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:02.493366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7368b1-f5f9-495b-a759-453a88c3d678 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:02.527734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:02.528362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:02.528639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:02.528907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:02.529521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:02.530974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc402fb-dee2-4f6a-a289-c01d32edb217 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:02.538825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 524/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:47:02 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:47:03.837329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06276-1eaa-4b9a-aab9-53391e64d1b3 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:03.837895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06276-1eaa-4b9a-aab9-53391e64d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:03.837895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac06276-1eaa-4b9a-aab9-53391e64d1b3 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:03.839509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 484/1008] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:03.844783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f372-be3e-49fc-8ef7-a6dc358daf61 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:03.845209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f372-be3e-49fc-8ef7-a6dc358daf61 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:03.845311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f372-be3e-49fc-8ef7-a6dc358daf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:03.864944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d492f372-be3e-49fc-8ef7-a6dc358daf61 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:03.866167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d492f372-be3e-49fc-8ef7-a6dc358daf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:03.889012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:03.955561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 525/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:04.223380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac06276-1eaa-4b9a-aab9-53391e64d1b3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:04.335187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac06276-1eaa-4b9a-aab9-53391e64d1b3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:04.336504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dafaf5-f184-4597-b88e-baa7472c18bd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.336789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dafaf5-f184-4597-b88e-baa7472c18bd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.337056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dafaf5-f184-4597-b88e-baa7472c18bd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.338415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 526/1010] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:04.343413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4872bab-64c2-43a4-9d72-301ba532b681 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.343695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4872bab-64c2-43a4-9d72-301ba532b681 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.343969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4872bab-64c2-43a4-9d72-301ba532b681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.363710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d4872bab-64c2-43a4-9d72-301ba532b681 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:04.365181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d4872bab-64c2-43a4-9d72-301ba532b681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:04.373391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:04.374053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.374329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.374599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.375703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:04.377120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:04.378717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 485/1011] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:04.384919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:04.429694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:04.438297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 527/1012] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:04.491038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd0728-7c69-46f0-b79c-2ee6e83124e5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:04.525592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:04.526436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.526857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.527270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.528077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:04.529636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:04.560356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 486/1013] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:47:04 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:47:04.701223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bdb79-248e-491d-8050-8d78d62debee None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.701582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bdb79-248e-491d-8050-8d78d62debee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.701809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bdb79-248e-491d-8050-8d78d62debee None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.703317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 528/1014] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:04.704546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:04.708662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.708938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.709211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.729364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:04.730608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:04.807727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:04.807933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 529/1015] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:04.818316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53eac2-6d85-4310-aae7-4be0d2050cb5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:04.853837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:04.854457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:04.854720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:04.854994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:04.855651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:04.857019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:04.875355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 487/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:47:04 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:05.085785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:05.102008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:05.189166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e03d34-df5c-4e1f-be39-40f5f32c836d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:05.229269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:05.229936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:05.230217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:05.230481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:05.231067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:05.232435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:05.242051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:05.243048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 07:47:05.243913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:05.245307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 530/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:47:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:05.249418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:05.358169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312475d0-ecbc-41d5-978b-ead43b656b3e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:05.393698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:05.394341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:05.394620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:05.394887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:05.396033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:05.396335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:05.397710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:05.407161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 488/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:47:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:05.408396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:05.485287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:05.513126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d5da9-9dd8-4db8-97bd-e0800232668e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:05.547581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:05.548189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:05.548462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:05.548726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:05.549353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:05.549635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:05.551088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6901841-9b47-4a91-b7bb-900ec40b5b94 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:05.560494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 531/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 07:47:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:05.561989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:05.581965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:05.665403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3628cb4-0f28-4329-abb9-6475cbc8575c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:05.698503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:05.699104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:05.699377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:05.699701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:05.700369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:05.701811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618fd4f-2b1f-458f-87a9-539093594d4f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:05.729778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 489/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:47:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:06.810649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d66d-ad44-4054-b924-64644e15b61a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:06.811223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d66d-ad44-4054-b924-64644e15b61a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:06.811223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788d66d-ad44-4054-b924-64644e15b61a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:06.812517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 532/1021] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:06.817994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e522f-6521-498e-be08-4780c4761bdb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:06.818395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e522f-6521-498e-be08-4780c4761bdb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:06.818537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e522f-6521-498e-be08-4780c4761bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:06.836862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-dd3e522f-6521-498e-be08-4780c4761bdb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:06.838071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dd3e522f-6521-498e-be08-4780c4761bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:06.909167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:06.916847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 490/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:07.084496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85132013-bf87-4bed-be67-5281f67a1fee None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.084693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85132013-bf87-4bed-be67-5281f67a1fee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.085161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85132013-bf87-4bed-be67-5281f67a1fee None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.086526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 533/1023] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:07.091990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.092273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.092539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.111238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:07.112438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:07.179572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 491/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:07.183760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85132013-bf87-4bed-be67-5281f67a1fee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.287932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85132013-bf87-4bed-be67-5281f67a1fee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:07.321867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:07.322771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.323254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.323724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.325018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:07.326696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:07.328796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 534/1025] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:07.334826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.444023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294b716-efc1-4c24-a290-9bbdf4f3cdcc None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:07.447248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.457310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:07.477051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:07.483673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:07.484288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.484560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.484831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.485479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:07.486942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:07.509498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 492/1026] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:47:07 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:47:07.562877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b76fa-d25d-442e-8f65-18e6c19a7e37 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:07.601224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:07.601827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.602109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.602383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.602985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:07.604419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:07.614169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:07.615248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:07.618795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 535/1027] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 07:47:07 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:07.621686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.732680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e2f1-ba62-47ab-a31f-4148acae6c96 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:07.768933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:07.769548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.770150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.770778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] RBAC: Adding query filter params (name=image) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:07.771086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:07.772434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:07.777627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 493/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:07.783339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.853455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:07.895505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a68d45-17d9-414e-821d-b444512995d4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:07.931860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:07.932477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.932756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.933025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.933648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:07.933980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:07.935601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8fca7-11aa-462e-81f3-804f9714adb8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:07.946051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 536/1029] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:07.947314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:07.978816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffbc27-7126-4cf3-94b6-0a130fa40294 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.979146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffbc27-7126-4cf3-94b6-0a130fa40294 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.979362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffbc27-7126-4cf3-94b6-0a130fa40294 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:07.980747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 537/1030] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:07.986307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:07.986307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:07.986307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.002258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:08.005361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:08.006585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:08.058147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938fc28-0d8d-4576-97d2-dbc98e457e09 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:08.076510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 538/1031] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:08.093806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:08.094424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:08.094704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:08.094974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.095676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] RBAC: Adding query filter params (interface=public, service_id=ed2efc5cd0384005a7c3d8f52269dc89, region_id=RegionOne) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:08.095982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:08.097352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:08.103312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 494/1032] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:47:07 2022] GET /identity/v3/endpoints?service_id=ed2efc5cd0384005a7c3d8f52269dc89&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:08.111996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:08.171681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:08.209592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f88fe6-8deb-41b5-936a-70bf5eaa8470 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:08.243810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:08.244416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:08.244698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:08.244967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.245567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:08.246963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:08.253528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 539/1033] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:47:08 2022] GET /identity/v3/services => generated 1957 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:47:08.352376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:08.464184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6ff1e-2f2d-4c92-9384-a0167eba7ea8 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:08.480412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:08.503294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:08.503932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:08.504215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:08.504515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.515529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:08.520895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:08.522070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:08.522923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:08.523917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 495/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:47:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:08.528243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:08.630077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb06df-8ed8-42ac-bc85-f7bd07c28993 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:08.656539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:08.666100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:08.666708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:08.666984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:08.667256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.667912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:08.668213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:08.669572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:08.677763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 540/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:47:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:08.683537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:08.787813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea18d6-31bc-4903-b57c-5ae5a751809f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:08.822040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:08.822678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:08.822958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:08.823227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:08.834398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:08.835834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:08.850096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:08.851038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:08.856749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:08.856749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:08.856749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:08.856749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 496/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:08.856974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:08.962765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3877a-7317-419c-9d83-c862fdde0093 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.000721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:09.001049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.001321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.001589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.002331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:09.002630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:09.003992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:09.016372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 541/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:09.022350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:09.051692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:09.126152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97690deb-a7bc-4f6a-8bd9-d9e5110c0d15 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.162766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:09.163399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.163673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.163940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.171455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:09.172876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:47:09.184649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:09.185578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:09.186563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 497/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:47:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:09.193986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:09.244726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:09.298731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e84dde-fd3b-4cc3-acd1-2199dd3a26ad None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.333163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:09.333703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.334009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.334276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.335000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:09.335303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:09.336728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:09.358146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 542/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:47:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:09.359816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:09.450129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:09.466564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e598302-e0eb-443a-bbb7-ee5f1cf69d44 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.499552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:09.500159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.500434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.500703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.566605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:09.568081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de404-4230-4272-871f-2a57c96a5fad None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:09.654364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:09.675581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 498/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:09.677889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:09.727388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6de516-e541-4250-a0cd-4653c362942a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.727696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6de516-e541-4250-a0cd-4653c362942a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.727941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6de516-e541-4250-a0cd-4653c362942a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.729408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 499/1041] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:09.734102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.734383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.734652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.752679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:09.753872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:09.786412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a4c8c-1b62-4cf6-a396-2cc60e1f971b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.796771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:09.825538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:09.826126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 500/1042] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:09.826267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:09.826507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:09.826777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:09.837109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:09.838557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:09.853372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:09.859615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:09.860845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:09.860845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:09.860845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:09.860845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 543/1043] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:09.862501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:09.971134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbcd64-9e62-4ddc-bd0f-7f69b73353ea None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:09.988999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.008669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.009283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.009563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.009866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.010514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:10.010819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.012186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.026055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 501/1044] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:10.030213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.100484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.135400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9719f76-5749-4a12-9139-c06a9cbb9bd1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.169744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.170393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.170668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.170931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.177817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.179253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:47:10.190496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:10.191399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:10.192382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 544/1045] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:10.197281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.213025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1509e34-5d23-4380-bca0-719f17bdf2b0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.220224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.249026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.249587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.249902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.250171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.261335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:10.266209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:10.267145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:10.268022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 502/1046] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:10.273979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.320654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.321984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b83c8-ac5b-488f-9d73-9e75cc1953ae None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.362912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.363521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.363797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.364065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.364751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:10.365051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.366545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.378711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 545/1047] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:10.385187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.394064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cde4e-dfd0-48cb-99ba-f342f4a6845f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.430119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.430730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.431394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.432155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:10.432472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.433862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0e3a7-bab6-4fdd-b552-ffc396df19a6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.444066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 503/1048] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:10.445126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7ee94-e2b7-4f62-88c8-055de4948d44 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.445410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7ee94-e2b7-4f62-88c8-055de4948d44 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.445686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7ee94-e2b7-4f62-88c8-055de4948d44 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.448810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 504/1049] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:10.449412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7ee94-e2b7-4f62-88c8-055de4948d44 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.469114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.491316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ddadf-27ad-4da9-8984-a4564ebd5934 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.525205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.525772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.526055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.526320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.527029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] RBAC: Adding query filter params (role.id=bc83cec3ef0247c2911858049a3f3e4c, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:10.527332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.528750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247c4d-130f-42ec-89a6-df76519eb7a9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.534019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 546/1050] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=bc83cec3ef0247c2911858049a3f3e4c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:10.535292 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.535577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.535844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.554727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:10.555950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:10.560724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.568222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7ee94-e2b7-4f62-88c8-055de4948d44 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.607797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.608566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.608847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.609118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] PATH_INFO: `/v3/users/46bb980f18424ebc9f00005a0dbcba59` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.624452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] RBAC: Authorizing `identity:get_user(user_id=46bb980f18424ebc9f00005a0dbcba59)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.626432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.630165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 505/1051] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/users/46bb980f18424ebc9f00005a0dbcba59 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:10.634852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.644686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 547/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:10.741781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da1c0e-9325-40d8-9018-69e4a3dc8159 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:10.776459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:10.776994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:10.777273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:10.777541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:10.802617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:10.846391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:10.847835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:10.912048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.960667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 506/1053] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 07:47:10.966281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:10.989880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:11.018595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0aeb2-f955-4107-bbc5-4d1e5ca79dcf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.025347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:11.061382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.062875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.063350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.063791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] PATH_INFO: `/v3/services/image` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.064528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.066057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.070284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbcbf-f797-44d2-86cc-b9da752827ef None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:11.076037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:11.077233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 07:47:11.078369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:11.080410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 548/1054] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:11.086560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:11.108398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.109028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.109306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.109578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.110725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.112139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.114484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 507/1055] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:10 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:11.120212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:11.194211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63671a59-0af6-4221-a270-1c744d3b29ad None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.227475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd923a-c304-44eb-8ee5-020d229d402f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.229933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.230549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.230831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.231105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.231741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:11.232041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.233421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.239859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 549/1056] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:11.245106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:11.247851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:11.255631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:11.266522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.267159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.267440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.267713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.268439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=6d23ff216b9146e7b693624315ce381a, user.id=46bb980f18424ebc9f00005a0dbcba59) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:11.268746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.270241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.275869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 508/1057] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:11 2022] GET /identity/v3/role_assignments?user.id=46bb980f18424ebc9f00005a0dbcba59&scope.project.id=6d23ff216b9146e7b693624315ce381a&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:11.359186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de9ee7-5bbd-4d21-8205-71ba4d5dc4f6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.396676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.397321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.397601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.397908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.398568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] RBAC: Adding query filter params (type=image) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:11.398871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.400315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9079424-90dc-4b96-9d59-61f0c174c6fe None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.411704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 550/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:11.413362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:11.517559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c1e32-7b56-4b53-b5fd-f558c88de4b7 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:11.554586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:11.555200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:11.555474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:11.555742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:11.556412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:11.557859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e0085-6042-4dbd-ac8c-290a38a0fb43 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:11.585768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 509/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:47:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:47:12.663293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed12-3c97-4dc5-9db2-8e9f1d3b34c4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:12.663817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed12-3c97-4dc5-9db2-8e9f1d3b34c4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:12.663817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819ed12-3c97-4dc5-9db2-8e9f1d3b34c4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:12.665187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 551/1060] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:12.670142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:12.670538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:12.670705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:12.689265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:12.690457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:12.730258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:12.764765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 510/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:13.034593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ed12-3c97-4dc5-9db2-8e9f1d3b34c4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.106395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:13.142767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819ed12-3c97-4dc5-9db2-8e9f1d3b34c4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.177300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:13.177958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.178237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.178508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.189325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:13.194050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:13.194963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:13.195811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 552/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:13.202435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.305007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64276420-0f7a-4819-b1c2-c1eb25861730 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.342348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:13.342955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.343226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.343493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.344208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:13.344509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:13.345895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa428db5-b7f8-4eab-a85e-b6b7a36c6747 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:13.358419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 511/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:13.359631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.412406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242cc9-23da-486a-98ba-c658131e6320 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.412736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242cc9-23da-486a-98ba-c658131e6320 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.412961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf242cc9-23da-486a-98ba-c658131e6320 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.414465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 512/1064] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:13.419269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.419552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.419822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.439075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:13.440615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:13.467234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef11c4b-903b-4bb8-8bf8-8bf1a86ccc49 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.467998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:13.496293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:13.506047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:13.506745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.506943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.507209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.514504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 513/1065] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:13.518602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:13.520631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:13.534828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:13.535792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:13.536725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:13.536725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:13.536725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:13.537422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 553/1066] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:13.541662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.645895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac306478-58d3-423b-947b-1f5d973f0850 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.682544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:13.683178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.683421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.683693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.684600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:13.684904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:13.686314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38521-492b-4da5-8823-c2eec3a30704 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:13.702499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 514/1067] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:13.703636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.756610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97045e-b5cd-4869-a71d-6a96fcc80ef4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.756907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97045e-b5cd-4869-a71d-6a96fcc80ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.757174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97045e-b5cd-4869-a71d-6a96fcc80ef4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.758775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 515/1068] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:47:13.764151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.764429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.764699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.783423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:13.784622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:13.801694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:13.813525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580669a6-5a4d-4787-b39e-d270764631a2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.846835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:13.847471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:13.847749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:13.848050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:13.851723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 516/1069] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:13.852366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.855627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:13.857068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:47:13.868779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:13.869914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:13.871087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:13.874847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 554/1070] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:13.875673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:13.921263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:13.968952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651332-9ddb-45a7-9293-d13a90b71824 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:13.983673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a3e85-cfa9-448f-af38-33fee62b987e None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.004751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.005388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.005661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.005955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.015821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:14.020612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:14.021537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:14.022421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:14.023058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.023058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.023220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 517/1071] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:14.023289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.023289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.023289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:14.023289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.024801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.029591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.044253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 555/1072] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:47:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:14.049511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.050036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.141152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849da6a-c97b-48b9-9a2e-cae08d0d02b5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.145242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.151597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670cde7-d380-45e4-b1e3-8c2a499b2599 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.174833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.175371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.175647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.175910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.177132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:14.177433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.178828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.185954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.186582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.186857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.187185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 518/1073] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:14.187288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.187971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.198463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.199893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:14.214142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:14.215083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:14.216005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:14.216005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:14.216005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:14.216005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 556/1074] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:14.216275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.263719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.288774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a869d-3b9e-4209-93d1-da8d26b475a0 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.314367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d90a2-4715-4739-97c4-1b5e8c051067 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.322315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.322918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.323192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.323458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.348272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.348896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.349174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.349447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] PATH_INFO: `/v3/users/46bb980f18424ebc9f00005a0dbcba59` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.350628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] RBAC: Authorizing `identity:get_user(user_id=46bb980f18424ebc9f00005a0dbcba59)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.352053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.353624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 557/1075] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/users/46bb980f18424ebc9f00005a0dbcba59 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:14.354402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.372358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.390144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.391614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.453703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7fc41-3bf5-4b76-aa44-7e0cd156685b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.463961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.486928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.487525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.487802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.488066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.489205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:14.489399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 519/1076] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:14.489520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.490240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.491048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.509103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 558/1077] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:14.509103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.581895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.591297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afec448-c354-4e9a-9442-4250e7ca10fe None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.604357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6cdea-fcbf-4338-9682-06ebe4a75152 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.624339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.624960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.625238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.625509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.626663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.628086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.630574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 520/1078] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:14.631215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.643220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.643847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.644213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.644486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.654250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.655663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:14.670242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:14.671418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:14.672607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:14.672607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:14.672607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:14.672607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 559/1079] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:14.672607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.675046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.736483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55513a-8227-464f-898c-047ffda04d63 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.770714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.771347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.771620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.771886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.772818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe3a8e-c1c4-4dec-93f1-99275fff8491 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.778452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.779914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:47:14.792089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:14.792982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:14.793918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 521/1080] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:14.794416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.803598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.812273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.812954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.813211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.813481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a/users/46bb980f18424ebc9f00005a0dbcba59/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.815390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d23ff216b9146e7b693624315ce381a, user_id=46bb980f18424ebc9f00005a0dbcba59, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.816805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.829053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 560/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:47:14 2022] PUT /identity/v3/projects/6d23ff216b9146e7b693624315ce381a/users/46bb980f18424ebc9f00005a0dbcba59/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:47:14.830423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:14.911433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779a51-a3ba-4b54-936c-b432e860f605 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.913662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:14.955275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:14.955904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:14.956186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:14.956455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:14.957128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:14.957472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:14.958929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:14.970546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1597d54-009f-4313-bdae-88bfde157351 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:14.981763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 522/1082] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:15.013864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:15.014423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:15.014697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:15.015043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:15.015768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:15.016129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:15.017602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a3505-6344-4493-885e-e5118e1b8a65 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:15.031479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 561/1083] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:15.038673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:15.156290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c627cc-b3fc-4b6c-b9b3-b175434ad50d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:15.192950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:15.194041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:15.194324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:15.194597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/bc83cec3ef0247c2911858049a3f3e4c` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:15.196787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=742f83bd38b542d193a911ef623abd45, role_id=bc83cec3ef0247c2911858049a3f3e4c)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:15.198257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b31b14-a18b-4473-ad62-09dba924cd31 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:15.213937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 523/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:47:15 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/742f83bd38b542d193a911ef623abd45/roles/bc83cec3ef0247c2911858049a3f3e4c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:47:17.034561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c6ca0-a107-41e9-81da-d3ff21c5e463 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.035141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c6ca0-a107-41e9-81da-d3ff21c5e463 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.035141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c6ca0-a107-41e9-81da-d3ff21c5e463 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.037151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 562/1085] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:17.042230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3bc5-b2f1-4d28-837c-cca4aeb83ee0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.042678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3bc5-b2f1-4d28-837c-cca4aeb83ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.042788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e3bc5-b2f1-4d28-837c-cca4aeb83ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.061149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c26e3bc5-b2f1-4d28-837c-cca4aeb83ee0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:17.062342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c26e3bc5-b2f1-4d28-837c-cca4aeb83ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:17.078773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:17.128163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 524/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:17.343843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7b43-3326-4671-87e4-1ff42e3428ff None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.344152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7b43-3326-4671-87e4-1ff42e3428ff None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.344429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe7b43-3326-4671-87e4-1ff42e3428ff None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.345790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 563/1087] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:17.351018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.351306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.351580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.369663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:17.370887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:17.391685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe7b43-3326-4671-87e4-1ff42e3428ff None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.437269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 525/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:17.442184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:17.492843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe7b43-3326-4671-87e4-1ff42e3428ff None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:17.525336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:17.525993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.526270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.526542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.536308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:17.541027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:17.542234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:17.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 564/1089] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:17.548152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.638532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:17.656320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211d4f5-f91f-4fc6-bfbd-2560df948a56 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:17.688915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:17.689514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.689819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.690089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.690742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:17.691039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:17.693005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:17.700766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 526/1090] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:17.706785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.724012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.805391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f5ad3-4582-44c8-9fd2-6017672e3ed4 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:17.814816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:17.828661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8222bf3-66cd-43dd-9437-01d1de68b220 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:17.840114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:17.840732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.841004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.841276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] PATH_INFO: `/v3/users/46bb980f18424ebc9f00005a0dbcba59` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.842400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] RBAC: Authorizing `identity:get_user(user_id=46bb980f18424ebc9f00005a0dbcba59)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:17.843829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:17.845377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 565/1091] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/users/46bb980f18424ebc9f00005a0dbcba59 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:17.851446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.862139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:17.862768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.863045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.863950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.874289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:17.879054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:17.880053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:17.880933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 527/1092] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:17.886211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:17.889795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:17.949605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39503496-2af6-4bc7-a3b1-d26a4f9b1516 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:17.982853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:17.983450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:17.983723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:17.983990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:17.986724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0860826-4079-454c-afc2-6570b635a44d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.033384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.034752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.036301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.036656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.036996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.038430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:18.038430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.039741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.051648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 528/1093] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:18.056843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.058024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.059731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.136676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.175638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6155a6f-727e-44ae-84d9-d301b626f31b None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.177698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 566/1094] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 07:47:18.183598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.215581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.216570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.217038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.217452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.228979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.230662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:18.247271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:18.248280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:18.249202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:18.249202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:18.249202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:18.255778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 529/1095] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:18.256796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.264576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.294596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0ab07-334d-47da-a08c-01803ddcf365 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.328939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.330325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.330600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.330868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.331980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.333942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.335546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 567/1096] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:18.341636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.359906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4231ec6-6bed-4f37-8829-6768422e3a7a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.376365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.395207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.395812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.396094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.396371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.397009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:18.397312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.398710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.410772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 530/1097] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:18.416359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.441041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294181f-af89-4bfa-b0da-1b990c427906 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.474955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.475559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.475838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.476104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.476823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=6d23ff216b9146e7b693624315ce381a, user.id=46bb980f18424ebc9f00005a0dbcba59) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:18.477126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.478586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.484110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 568/1098] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/role_assignments?user.id=46bb980f18424ebc9f00005a0dbcba59&scope.project.id=6d23ff216b9146e7b693624315ce381a&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:18.514650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae426f1c-7e26-4b36-90ff-dde759a65a9d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.552678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.553322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.553597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.553897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.562226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.563682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:47:18.577278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:18.578378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:18.579382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 531/1099] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:18.585144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.596844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.687208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b540d3-2195-46c3-bb64-050db861325c None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.720598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.721128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.721402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.721665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.722410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:18.722836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.725394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.747069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 569/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:18.752947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:18.792196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:18.851122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470a1cf-ab03-48ca-a74e-d1fde052274f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:18.883484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:18.884097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:18.884373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:18.884643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:18.953900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:18.955334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:18.992722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:19.057824 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 532/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:19.064038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:19.128199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:19.162396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1b002-bb6e-43f5-85d6-72b01c7a43e9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:19.196328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:19.196956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:19.197350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:19.197625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:19.207475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:19.208895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:19.222861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:19.223982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:47:19.224916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:19.224916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:47:19.224916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:19.224916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 570/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:19.229713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:19.330450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8353498-9dc9-4515-80fd-5435d4510b71 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:19.373810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:19.374427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:19.374707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:19.374976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:19.375781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:19.376230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:19.377867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:19.385779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:19.397232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 533/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:19.398480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:19.499608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37862986-fcde-4a73-8692-52ac306694d2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:19.515611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:19.534543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:19.535702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:19.535983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:19.536256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:19.542772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:19.544209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:47:19.555898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:19.556951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:19.557924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 571/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:47:19.563375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:19.670563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eba00d-7903-4157-8b45-e9affd0280e4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:19.704397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:19.705002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:19.705278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:19.705547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:19.706274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:19.706580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:19.708507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfbae1-40f2-4172-a955-853f04529961 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:19.720277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 534/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:19.727219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:19.823244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e0fb-84dc-4e9e-a3aa-b9e5f09390db None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:19.855953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:19.856548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:19.856825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:19.857094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:19.857837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] RBAC: Adding query filter params (role.id=bc83cec3ef0247c2911858049a3f3e4c, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=742f83bd38b542d193a911ef623abd45) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:19.858146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:19.859571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedda52-2d7c-4b8c-bf87-38043b836720 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:19.865091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 572/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:19 2022] GET /identity/v3/role_assignments?user.id=742f83bd38b542d193a911ef623abd45&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=bc83cec3ef0247c2911858049a3f3e4c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:20.674953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9432-ac13-444d-bb4f-121e18ae1577 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:20.678153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9432-ac13-444d-bb4f-121e18ae1577 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:20.678153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9432-ac13-444d-bb4f-121e18ae1577 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:20.678153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 535/1107] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:20.683136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:20.683601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:20.683703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:20.704087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:20.705262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:20.741595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:20.771900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 573/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:21.041602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9432-ac13-444d-bb4f-121e18ae1577 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:21.099275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:21.140840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9432-ac13-444d-bb4f-121e18ae1577 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:21.173493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:21.174134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:21.174411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:21.174681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:21.176258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:21.177671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08a0d3-e484-4a12-8db9-7fd325cf594c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:21.179241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 536/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:21.185141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:21.279891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad48100-a904-44ed-9368-2a166b849d98 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:21.312782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:21.313323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:21.313602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:21.313903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:21.314550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:21.315979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c677de-4779-45aa-ae40-0791ceb2f115 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:21.337839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 574/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:47:21 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:23.502253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9cd3-bf62-41c8-a443-c0b6447e56ee None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:23.502809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9cd3-bf62-41c8-a443-c0b6447e56ee None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:23.502809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9cd3-bf62-41c8-a443-c0b6447e56ee None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:23.504169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 537/1111] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:23.509418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985b71-12d1-44b3-a707-a9908265892e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:23.509850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985b71-12d1-44b3-a707-a9908265892e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:23.509987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65985b71-12d1-44b3-a707-a9908265892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:23.528079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-65985b71-12d1-44b3-a707-a9908265892e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:23.529247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-65985b71-12d1-44b3-a707-a9908265892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:23.552349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:23.596672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 575/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:23.871773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9cd3-bf62-41c8-a443-c0b6447e56ee None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:23.973412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9cd3-bf62-41c8-a443-c0b6447e56ee None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:24.003237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:24.014254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:24.014885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:24.015162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:24.015432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] PATH_INFO: `/v3/roles/b1177515548b419b92d3359f27435e02` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:24.016817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] RBAC: Authorizing `identity:get_role(role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:24.018219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:24.020196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 538/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:23 2022] GET /identity/v3/roles/b1177515548b419b92d3359f27435e02 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:24.025990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985b71-12d1-44b3-a707-a9908265892e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:24.124197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65985b71-12d1-44b3-a707-a9908265892e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:24.157907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:24.158535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:24.158809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:24.159078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] PATH_INFO: `/v3/users/373ad95b52694afabb9ec8502e83d700` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:24.171587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] RBAC: Authorizing `identity:get_user(user_id=373ad95b52694afabb9ec8502e83d700)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:24.173013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:24.174617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 576/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:24 2022] GET /identity/v3/users/373ad95b52694afabb9ec8502e83d700 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:24.181184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:24.280024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a845-c271-4c38-93a0-d3787bed4344 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:24.313812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:24.314410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:24.314687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:24.314956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:24.373891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:24.381617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:24.383024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:24.483438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 539/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:24.489517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:24.507626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:24.589577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e625d-f4c7-407a-bc2d-278490e94d5b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:24.624484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:24.625101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:24.625373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:24.625638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:24.626770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:24.628186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9878e-1dc3-4657-ab18-d79a94ca150f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:24.630336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 577/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:24 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:24.636773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:24.730365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:24.737240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243720dc-e38f-4dda-a376-c91bfde1fae6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:24.770529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:24.771120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:24.771394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:24.771661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:24.772373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=6d23ff216b9146e7b693624315ce381a, user.id=373ad95b52694afabb9ec8502e83d700) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:24.772675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:24.774144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24f7d4-d19a-46b8-8d59-249427a9851d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:24.779317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 540/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:24 2022] GET /identity/v3/role_assignments?user.id=373ad95b52694afabb9ec8502e83d700&scope.project.id=6d23ff216b9146e7b693624315ce381a&role.id=b1177515548b419b92d3359f27435e02 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:26.884316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02213da-adfc-410e-81dd-c36f318f84da None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:26.884847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02213da-adfc-410e-81dd-c36f318f84da None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:26.884847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02213da-adfc-410e-81dd-c36f318f84da None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:26.886215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 578/1118] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:26.891477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:26.891851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:26.892020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:26.910614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:26.911798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:26.975375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 541/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:27.241532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02213da-adfc-410e-81dd-c36f318f84da None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:27.322965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:27.340012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02213da-adfc-410e-81dd-c36f318f84da None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:27.372835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:27.373450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:27.373752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:27.374027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] PATH_INFO: `/v3/roles/b1177515548b419b92d3359f27435e02` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:27.375403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] RBAC: Authorizing `identity:get_role(role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:27.376756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:27.378073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 579/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:27 2022] GET /identity/v3/roles/b1177515548b419b92d3359f27435e02 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:27.383690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:27.467110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:27.479736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e9a99-bd83-4ed7-8154-0fa062327772 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:27.511774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:27.512335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:27.512612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:27.512884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] PATH_INFO: `/v3/users/373ad95b52694afabb9ec8502e83d700` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:27.514043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] RBAC: Authorizing `identity:get_user(user_id=373ad95b52694afabb9ec8502e83d700)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:27.515494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:27.517044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 542/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:27 2022] GET /identity/v3/users/373ad95b52694afabb9ec8502e83d700 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:27.523239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:27.618503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30d9a9-6197-4e80-a4af-2f00bf2badd3 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:27.651407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646875b-2818-4887-a799-7d282a913702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:27.652019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646875b-2818-4887-a799-7d282a913702 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:27.652293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646875b-2818-4887-a799-7d282a913702 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:27.652566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646875b-2818-4887-a799-7d282a913702 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:27.653664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646875b-2818-4887-a799-7d282a913702 None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:27.655098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646875b-2818-4887-a799-7d282a913702 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:27.656659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 580/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:27 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:27.662662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:27.759134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e4a4-6494-4434-8603-f6b7994058e2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:27.792442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:27.793082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:27.793357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:27.793627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a/users/373ad95b52694afabb9ec8502e83d700/roles/b1177515548b419b92d3359f27435e02` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:27.795770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d23ff216b9146e7b693624315ce381a, user_id=373ad95b52694afabb9ec8502e83d700, role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:27.797191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bfaee-7f08-4175-9c3e-db1cfdb0d408 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:27.809521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 543/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:47:27 2022] PUT /identity/v3/projects/6d23ff216b9146e7b693624315ce381a/users/373ad95b52694afabb9ec8502e83d700/roles/b1177515548b419b92d3359f27435e02 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:47:29.897671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0ebc-d914-4b06-bacd-9054cc8cb0b9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:29.898148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0ebc-d914-4b06-bacd-9054cc8cb0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:29.898247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0ebc-d914-4b06-bacd-9054cc8cb0b9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:29.899577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 581/1124] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:29.904776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:29.905176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:29.905325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:29.923591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:29.924770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:29.936122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:29.991896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 544/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:30.265533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f0ebc-d914-4b06-bacd-9054cc8cb0b9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:30.287871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:30.366001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f0ebc-d914-4b06-bacd-9054cc8cb0b9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:30.398142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:30.398764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:30.399040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:30.399309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] PATH_INFO: `/v3/roles/b1177515548b419b92d3359f27435e02` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:30.400685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] RBAC: Authorizing `identity:get_role(role_id=b1177515548b419b92d3359f27435e02)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:30.402078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:30.403362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 582/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:30 2022] GET /identity/v3/roles/b1177515548b419b92d3359f27435e02 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:30.409303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:30.504950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c4454-fffd-46ca-a909-badf5966aaa3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:30.540795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:30.541437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:30.541716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:30.542014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] PATH_INFO: `/v3/users/373ad95b52694afabb9ec8502e83d700` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:30.543143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] RBAC: Authorizing `identity:get_user(user_id=373ad95b52694afabb9ec8502e83d700)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:30.544572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:30.546195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 545/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:30 2022] GET /identity/v3/users/373ad95b52694afabb9ec8502e83d700 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:30.552112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:30.636929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:30.650053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1e45-2927-4d08-a260-a55bc892f2a6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:30.682392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:30.682987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:30.683264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:30.683533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:30.748799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:30.750259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:30.834101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:30.847081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 583/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:30.853012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:30.868158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:30.951388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c715537-0ffa-402e-bc54-b36fa5d394ba None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:30.983329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:30.983944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:30.984220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:30.984490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] PATH_INFO: `/v3/projects/6d23ff216b9146e7b693624315ce381a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:30.985581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] RBAC: Authorizing `identity:get_project(project_id=6d23ff216b9146e7b693624315ce381a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:30.987059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933efeb-3393-4f97-a775-875fdb4e3334 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:30.989146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 546/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:30 2022] GET /identity/v3/projects/6d23ff216b9146e7b693624315ce381a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:30.995369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:31.090206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375841c-bf54-4317-ac45-fa9658a1e9ed None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:31.123346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:31.123936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:31.124214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:31.124481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:31.125198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] RBAC: Adding query filter params (role.id=b1177515548b419b92d3359f27435e02, scope.project.id=6d23ff216b9146e7b693624315ce381a, user.id=373ad95b52694afabb9ec8502e83d700) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:31.125501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:31.126974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b69f-d227-4a82-b24b-c7fdbd7d6ae9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:31.134417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 584/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:30 2022] GET /identity/v3/role_assignments?user.id=373ad95b52694afabb9ec8502e83d700&scope.project.id=6d23ff216b9146e7b693624315ce381a&role.id=b1177515548b419b92d3359f27435e02 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:33.267765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922caf8-6dd4-439c-a0fe-d31108590d7c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:33.268267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922caf8-6dd4-439c-a0fe-d31108590d7c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:33.268267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922caf8-6dd4-439c-a0fe-d31108590d7c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:33.269576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 547/1131] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:33.274674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:33.275065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:33.275212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:33.292823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:33.294019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:33.339153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:33.358304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 585/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:33.628702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922caf8-6dd4-439c-a0fe-d31108590d7c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:33.707552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:33.726232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922caf8-6dd4-439c-a0fe-d31108590d7c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:33.758801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:33.759420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:33.759693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:33.759958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:33.761043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:33.762486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a7861-3230-48dd-87be-17b7c3235e78 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:33.764009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 548/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:33.769803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:33.863187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1eae7-6db4-4c04-98ce-8539abb66a73 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:33.895324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:33.895924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:33.896198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:33.896465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:33.897122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:33.898595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39378452-db7e-45fc-9ad8-51d99275944d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:33.924124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 586/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:47:33 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:36.081905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4518-cd1c-4281-a335-53193528f5f6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:36.082448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4518-cd1c-4281-a335-53193528f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:36.082448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4518-cd1c-4281-a335-53193528f5f6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:36.083775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 549/1135] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:36.089041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:36.089449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:36.089629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:36.107952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:36.109146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:36.135443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:36.175427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 587/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:36.441442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4518-cd1c-4281-a335-53193528f5f6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:36.539219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4518-cd1c-4281-a335-53193528f5f6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:36.572753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:36.573368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:36.573642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:36.573938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:36.575030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:36.576449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf8196-e7f4-4086-93e9-79540b0b372a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:36.578023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 550/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:47:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:36.584245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:36.678453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc84f3-b0a4-40e1-9429-bf30e8c98478 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:36.712342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:36.712875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:36.713147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:36.713410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:36.714150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:36.716043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d837-005e-4ed8-a571-634977b6ad5f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:36.735909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 588/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:47:36 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:38.888556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110e7f2-39b9-4d6e-9356-496ea557b91a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:38.888950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110e7f2-39b9-4d6e-9356-496ea557b91a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:38.889071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110e7f2-39b9-4d6e-9356-496ea557b91a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:38.890461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 551/1139] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:38.895802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:38.896222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:38.896419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:38.914540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:38.915710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:38.929710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:38.980961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 589/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:39.253100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110e7f2-39b9-4d6e-9356-496ea557b91a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:39.272013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:39.352572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110e7f2-39b9-4d6e-9356-496ea557b91a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:39.385174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:39.385810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:39.386091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:39.386362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:39.396567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:39.401263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:39.402403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:39.403361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 552/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:39.408297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:39.502661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf63235-6dec-4bd8-a282-c0e5f59e3e75 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:39.535482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:39.536021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:39.536295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:39.536563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:39.537217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:39.537520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:39.538906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:39.546541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 590/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:39.552587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:39.628471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:39.649561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69e3f8-ad0a-4c69-96a4-075b08d888de None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:39.680958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:39.681571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:39.681881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:39.682147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] PATH_INFO: `/v3/users/b4951bb3742b47258c31466cbcdb458e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:39.695450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] RBAC: Authorizing `identity:get_user(user_id=b4951bb3742b47258c31466cbcdb458e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:39.696864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:39.698487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 553/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:39 2022] GET /identity/v3/users/b4951bb3742b47258c31466cbcdb458e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:47:39.704269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:39.719824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:39.800826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5f066-f15b-4442-a92d-7e5c34b8c988 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:39.834507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:39.835395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:39.835778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:39.836054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:39.906191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:39.907637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:39.920649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:40.002169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 591/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:40.008610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:40.102739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59b43e-b477-4761-b229-1b724d14ed3f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:40.136456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:40.137080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:40.137355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:40.137624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] PATH_INFO: `/v3/projects/04b8b05091564b9189489c8c26ed58f2` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:40.138759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] RBAC: Authorizing `identity:get_project(project_id=04b8b05091564b9189489c8c26ed58f2)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:40.140179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdb808-2f66-44eb-a1e8-16c6cbbdf43e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:40.141897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 554/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:40 2022] GET /identity/v3/projects/04b8b05091564b9189489c8c26ed58f2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:40.147850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:40.242366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dede75b-77ac-4f88-b3dd-9bcdb697d265 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:40.255306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:40.275671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:40.276263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:40.276539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:40.276809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:40.277521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=04b8b05091564b9189489c8c26ed58f2, user.id=b4951bb3742b47258c31466cbcdb458e) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:40.277857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:40.279267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68945b5f-f44b-46e6-9940-d410b4f5727e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:40.284928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 592/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:40 2022] GET /identity/v3/role_assignments?user.id=b4951bb3742b47258c31466cbcdb458e&scope.project.id=04b8b05091564b9189489c8c26ed58f2&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:42.397664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52c09-0568-4aa3-b89e-32a602529cc4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:42.398221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52c09-0568-4aa3-b89e-32a602529cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:42.398221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52c09-0568-4aa3-b89e-32a602529cc4 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:42.399547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 555/1147] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:42.404557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:42.404986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:42.405088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:42.424071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:42.425250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:42.488894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 593/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:42.755512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52c09-0568-4aa3-b89e-32a602529cc4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:42.759271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:42.855222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52c09-0568-4aa3-b89e-32a602529cc4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:42.887779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:42.888407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:42.888685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:42.888955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:42.899680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:42.904446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:42.905644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:42.906733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 556/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:42.911759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:43.007135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4069f-bbf9-48fe-a5d5-9fd7067aa8c2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:43.014057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:43.039648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:43.040255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:43.040531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:43.040798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:43.041454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:43.041784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:43.043150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:43.051353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 594/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:43.057092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:43.117324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:43.153853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ddd9b-841c-4c79-922a-f31dbd427080 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:43.186424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:43.187036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:43.187309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:43.187577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] PATH_INFO: `/v3/users/b4951bb3742b47258c31466cbcdb458e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:43.188677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] RBAC: Authorizing `identity:get_user(user_id=b4951bb3742b47258c31466cbcdb458e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:43.190707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:43.192231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 557/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:43 2022] GET /identity/v3/users/b4951bb3742b47258c31466cbcdb458e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:43.197842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:43.292694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65978-405c-41d0-b978-ad5d6485be28 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:43.324768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:43.325377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:43.325649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:43.325946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] PATH_INFO: `/v3/projects/04b8b05091564b9189489c8c26ed58f2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:43.327037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] RBAC: Authorizing `identity:get_project(project_id=04b8b05091564b9189489c8c26ed58f2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:43.328541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce738a-fca5-43a1-97a2-0d407cdf03fd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:43.330129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 595/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:43 2022] GET /identity/v3/projects/04b8b05091564b9189489c8c26ed58f2 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:43.335755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:43.430706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271692f-1cde-4c99-adf3-d36b632612d6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:43.459198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:43.464299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:43.464939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:43.465213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:43.465480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] PATH_INFO: `/v3/projects/04b8b05091564b9189489c8c26ed58f2/users/b4951bb3742b47258c31466cbcdb458e/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:43.467371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] RBAC: Authorizing `identity:create_grant(project_id=04b8b05091564b9189489c8c26ed58f2, user_id=b4951bb3742b47258c31466cbcdb458e, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:43.468766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc777b5-499b-4159-a4dd-43eac4ff4405 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:43.481410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 558/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:47:43 2022] PUT /identity/v3/projects/04b8b05091564b9189489c8c26ed58f2/users/b4951bb3742b47258c31466cbcdb458e/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:47:45.561287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eec79-d501-49e2-8626-da8d6d67d762 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:45.561846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eec79-d501-49e2-8626-da8d6d67d762 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:45.561846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eec79-d501-49e2-8626-da8d6d67d762 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:45.563197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 596/1154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:45.568359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:45.568748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:45.568892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:45.586590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:45.587750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:45.652000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 559/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:45.917248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8eec79-d501-49e2-8626-da8d6d67d762 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:45.952972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:46.018509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8eec79-d501-49e2-8626-da8d6d67d762 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.051094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.051710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.051983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.052250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.062781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:47:46.067467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:46.068516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:47:46.069431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 597/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:47:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:46.074656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:46.170526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ec80f-e8a0-4872-ba92-a864fd6714d0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.193803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:46.205959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.206563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.206842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.207114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.207776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:46.208077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:46.209438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:46.217258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 560/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:46.223045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:46.314504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:46.322206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef107d-09bf-46a4-bde0-fc1efab72820 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.355210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.355832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.356111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.356386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] PATH_INFO: `/v3/users/b4951bb3742b47258c31466cbcdb458e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.357514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] RBAC: Authorizing `identity:get_user(user_id=b4951bb3742b47258c31466cbcdb458e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:46.358981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:46.360542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 598/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:47:46 2022] GET /identity/v3/users/b4951bb3742b47258c31466cbcdb458e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:46.366885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:46.462439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfc7ce-14b8-484b-a0b8-a29282d7ae0c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.495270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.495857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.496131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.496397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.545257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:46.561841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:46.563267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:46.660699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 561/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:46.666726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:46.762540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f57cd-8ba0-4b19-9ad7-d273fb8ead8b None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.795661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.795661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.795661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.795989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] PATH_INFO: `/v3/projects/04b8b05091564b9189489c8c26ed58f2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.797019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=04b8b05091564b9189489c8c26ed58f2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:46.798466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd809eb-3fdd-475c-aa79-38ef67f65bb1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:46.800090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 599/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:47:46 2022] GET /identity/v3/projects/04b8b05091564b9189489c8c26ed58f2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:46.806128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:46.887941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:46.904027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59296e-89dd-4441-bc62-13db7e705d8f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:46.937348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:46.937972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:46.938251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:46.938520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:46.939248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=04b8b05091564b9189489c8c26ed58f2, user.id=b4951bb3742b47258c31466cbcdb458e) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:46.939552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:46.941007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63149b-b746-4d2f-beff-27e43c3c0f0e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:46.946594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 562/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:47:46 2022] GET /identity/v3/role_assignments?user.id=b4951bb3742b47258c31466cbcdb458e&scope.project.id=04b8b05091564b9189489c8c26ed58f2&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:49.121944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2354-16af-4c84-b684-768194d5dcea None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:49.122512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2354-16af-4c84-b684-768194d5dcea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:49.122512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2354-16af-4c84-b684-768194d5dcea None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:49.123782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 600/1162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:49.129541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:49.130001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:49.130127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:49.148484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:49.149589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:49.212999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 563/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:49.476417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2354-16af-4c84-b684-768194d5dcea None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:49.507033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:49.576137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2354-16af-4c84-b684-768194d5dcea None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:49.612187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:49.612799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:49.613081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:49.613354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:49.682537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:49.684232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:49.713061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:49.779486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 601/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:47:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:47:49.786074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:49.860312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:49.883608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97b1e4-671d-491f-8fb0-45a301c8168a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:49.917713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:49.918368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:49.918646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:49.918917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:49.925386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:49.926843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-63092417-f1eb-49ad-9c2f-65af19cf4f75 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:47:49.938181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:47:49.939367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:47:49.940393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 564/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:47:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:47:49.945621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:50.047553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65187d2-905e-4306-bd48-a22523658157 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:50.076314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:50.082286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:50.082916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:50.083189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:50.083458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:50.084092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:47:50.084390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:50.085781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0df4ea-8ab3-4f39-a8e7-616a452d5475 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:50.092703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 602/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 07:47:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:52.236227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a3110-67e4-4032-96ae-0e8feaa8e27e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:52.237223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a3110-67e4-4032-96ae-0e8feaa8e27e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:52.237223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a3110-67e4-4032-96ae-0e8feaa8e27e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:52.238547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 565/1167] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:52.243846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3163a6d-f9af-47ac-8e81-fc4d91783800 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:52.244273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3163a6d-f9af-47ac-8e81-fc4d91783800 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:52.244373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3163a6d-f9af-47ac-8e81-fc4d91783800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:52.262740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3163a6d-f9af-47ac-8e81-fc4d91783800 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:52.263910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3163a6d-f9af-47ac-8e81-fc4d91783800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:52.328426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 603/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:47:52.595503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a3110-67e4-4032-96ae-0e8feaa8e27e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:52.700294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a3110-67e4-4032-96ae-0e8feaa8e27e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:52.733330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:52.733959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:52.734242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:52.734513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:52.735170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:52.736523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c04c7-d2bf-4cc0-8644-8fb2b01ebac0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:52.745686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:52.753839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 566/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Tue Sep 13 07:47:52 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:54.942186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ff3-d79f-4e1c-91b2-da4500ccb746 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:54.942721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ff3-d79f-4e1c-91b2-da4500ccb746 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:54.942721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd3ff3-d79f-4e1c-91b2-da4500ccb746 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:54.944066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 604/1170] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:54.949363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:54.949758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:54.949933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:54.967596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:54.968775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:55.032700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 567/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:55.312854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3ff3-d79f-4e1c-91b2-da4500ccb746 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:55.416948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd3ff3-d79f-4e1c-91b2-da4500ccb746 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:55.441956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:55.450840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:55.451463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:55.451741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:55.452012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] PATH_INFO: `/v3/domains/4e6221b5c7a94f1d9d2666b6ab5937e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:55.462176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=4e6221b5c7a94f1d9d2666b6ab5937e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:55.463615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013c611-7fa5-41dc-b6c7-dfc19a8fe4eb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:55.465183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 605/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Sep 13 07:47:55 2022] GET /identity/v3/domains/4e6221b5c7a94f1d9d2666b6ab5937e3 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:55.471559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:55.567455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b736e6-dbfb-4e7a-aa2f-244906c1e093 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:55.592534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:55.600905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:55.601515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:55.601828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:55.602102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:55.602752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:55.604171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787b59f-846a-4909-b9c7-7fddac7dda70 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:55.629809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 568/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:47:55 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:47:57.818297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bae5f-89ca-48b2-992a-523f8e3458d8 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:57.818832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bae5f-89ca-48b2-992a-523f8e3458d8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:57.818832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bae5f-89ca-48b2-992a-523f8e3458d8 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:57.820147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 606/1174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:47:57.825306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:57.825764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:57.825877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:57.844429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:47:57.845596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:47:57.911483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 569/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:47:58.183022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bae5f-89ca-48b2-992a-523f8e3458d8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:58.280969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bae5f-89ca-48b2-992a-523f8e3458d8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:58.312320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:58.312926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:58.313206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:58.313468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] PATH_INFO: `/v3/domains/4e6221b5c7a94f1d9d2666b6ab5937e3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:58.315070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4e6221b5c7a94f1d9d2666b6ab5937e3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:58.316481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd000f11-034e-480c-b28a-54e53a71d440 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:58.318031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 607/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Sep 13 07:47:58 2022] GET /identity/v3/domains/4e6221b5c7a94f1d9d2666b6ab5937e3 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:47:58.323569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:47:58.419508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a68b9-37a6-4853-aff4-681611c29fba None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:47:58.430161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:47:58.454018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:47:58.454540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:47:58.454820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:47:58.455087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:47:58.456343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:47:58.457816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc04614-5574-47ef-abf6-c13f94a4a448 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:47:58.477855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 570/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:47:58 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:48:00.617497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa652-7fd7-4785-a610-29de136f4a63 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:00.618084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa652-7fd7-4785-a610-29de136f4a63 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:00.618084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fa652-7fd7-4785-a610-29de136f4a63 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:00.619391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 608/1178] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:48:00.624747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:00.625162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:00.625294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:00.644220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:48:00.645400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:48:00.710117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 571/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:48:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:48:00.978281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa652-7fd7-4785-a610-29de136f4a63 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.033635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:01.082338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fa652-7fd7-4785-a610-29de136f4a63 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:01.116416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:01.117039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:01.117318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:01.117591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:01.128542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:48:01.133268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:48:01.134425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:48:01.135474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 609/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:48:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:48:01.140500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.230460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:01.237354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c69a0b-1358-4c40-970e-fcd57f51a339 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:01.270641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:01.271235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:01.271509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:01.271779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:01.272430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:48:01.272729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:01.274112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:01.281833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 572/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:01.288151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.384344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c3547-2bb2-4d84-a4eb-8f9fb101f96f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:01.397308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:01.418591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:01.419221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:01.419497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:01.419765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] PATH_INFO: `/v3/users/be00e8c424a0454d91354428fd7d6976` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:01.434571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] RBAC: Authorizing `identity:get_user(user_id=be00e8c424a0454d91354428fd7d6976)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:01.436022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:01.437690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 610/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:48:01 2022] GET /identity/v3/users/be00e8c424a0454d91354428fd7d6976 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:01.443545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.540453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2387ba-20cf-4e04-88d1-79b110a3188d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:01.573477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:01.574101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:01.574376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:01.574642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:01.600160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:01.639803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:01.641240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:01.741875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 573/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:48:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:48:01.747814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.843610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35778a0-1a18-4a03-bae1-d8bb400807bb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:01.876198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:01.876836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:01.877116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:01.877385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] PATH_INFO: `/v3/projects/2795fcb5e37c4ba6a636229feebd5304` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:01.878520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] RBAC: Authorizing `identity:get_project(project_id=2795fcb5e37c4ba6a636229feebd5304)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:01.879937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47447ce-a855-4775-b45d-4712fbd32708 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:01.881506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 611/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:48:01 2022] GET /identity/v3/projects/2795fcb5e37c4ba6a636229feebd5304 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:01.887699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:01.944309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:01.985370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6da27-07ef-4440-9d85-95caf565d8b6 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:02.019120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:02.019666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:02.019946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:02.020219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:02.020973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2795fcb5e37c4ba6a636229feebd5304, user.id=be00e8c424a0454d91354428fd7d6976) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:48:02.021279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:02.022748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e0fe-0e80-4aa0-a799-f7a219fff3a0 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:02.030595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 574/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:48:01 2022] GET /identity/v3/role_assignments?user.id=be00e8c424a0454d91354428fd7d6976&scope.project.id=2795fcb5e37c4ba6a636229feebd5304&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:04.156778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24882eda-3d9a-40ea-a650-5771166dc790 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:04.157301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24882eda-3d9a-40ea-a650-5771166dc790 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:04.157301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24882eda-3d9a-40ea-a650-5771166dc790 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:04.159367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 612/1186] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:48:04.164456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:04.164831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:04.165012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:04.182839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:48:04.184272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:48:04.248194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 575/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:48:04.527075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24882eda-3d9a-40ea-a650-5771166dc790 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:04.568797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:04.624410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24882eda-3d9a-40ea-a650-5771166dc790 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:04.657179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:04.657821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:04.658125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:04.658394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:04.668223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:48:04.672942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:48:04.674584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:48:04.675467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 613/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:48:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:48:04.680546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:04.730594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:04.775851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b37a9-313f-48c2-81e2-2db3db5a375f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:04.808733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:04.809329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:04.809605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:04.809900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:04.810555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:48:04.810854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:04.812202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:04.820421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 576/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:04.826151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:04.921416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:04.925049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167a1b-fee5-49ab-a6a6-18ecff877f30 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:04.958453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:04.959065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:04.959339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:04.959606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] PATH_INFO: `/v3/users/be00e8c424a0454d91354428fd7d6976` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:04.960717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] RBAC: Authorizing `identity:get_user(user_id=be00e8c424a0454d91354428fd7d6976)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:04.962164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:04.963710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 614/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:48:04 2022] GET /identity/v3/users/be00e8c424a0454d91354428fd7d6976 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:04.969527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:05.067204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ce31d-a3f2-43b0-ba43-01ff26f30cc4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:05.080252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:05.100781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:05.101406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:05.101684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:05.101984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] PATH_INFO: `/v3/projects/2795fcb5e37c4ba6a636229feebd5304` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:05.103088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] RBAC: Authorizing `identity:get_project(project_id=2795fcb5e37c4ba6a636229feebd5304)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:05.104518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eaba26-2382-413b-ab3a-cab7fce7a915 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:05.106175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 577/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:48:04 2022] GET /identity/v3/projects/2795fcb5e37c4ba6a636229feebd5304 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:05.112797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:05.206641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a560c14-457c-4bcd-8de4-2a23ba06303f None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:05.239350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:05.239985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:05.240260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:05.240527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] PATH_INFO: `/v3/projects/2795fcb5e37c4ba6a636229feebd5304/users/be00e8c424a0454d91354428fd7d6976/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:05.242403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] RBAC: Authorizing `identity:create_grant(project_id=2795fcb5e37c4ba6a636229feebd5304, user_id=be00e8c424a0454d91354428fd7d6976, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:05.243812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0ca9a-8681-42b3-8bbd-9d7885b7fd75 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:05.255963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 615/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:48:05 2022] PUT /identity/v3/projects/2795fcb5e37c4ba6a636229feebd5304/users/be00e8c424a0454d91354428fd7d6976/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:48:07.367034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65e24-50ab-4ae0-a1a3-b971d6ed7787 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:07.367563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65e24-50ab-4ae0-a1a3-b971d6ed7787 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:07.367563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65e24-50ab-4ae0-a1a3-b971d6ed7787 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:07.368889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 578/1193] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:48:07.374155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:07.374587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:07.374677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:07.393290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:48:07.394486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:48:07.415974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:07.459288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 616/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:48:07.727575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c65e24-50ab-4ae0-a1a3-b971d6ed7787 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:07.827331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c65e24-50ab-4ae0-a1a3-b971d6ed7787 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:07.858782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:07.859390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:07.859663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:07.859930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:07.870537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:48:07.875190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:48:07.876257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:48:07.877700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 579/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:48:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:48:07.882762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:07.976975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f0db-9fbe-4264-aa57-0383d666a688 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:08.009688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:08.010236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:08.010508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:08.010772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:08.011418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:48:08.011715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:08.013067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:08.020702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 617/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:48:08.026591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:08.069920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:08.124481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37044a6-9cc6-4cb6-9971-d8aa58f5c803 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:08.156247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:08.156859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:08.157131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:08.157397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] PATH_INFO: `/v3/users/be00e8c424a0454d91354428fd7d6976` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:08.159027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] RBAC: Authorizing `identity:get_user(user_id=be00e8c424a0454d91354428fd7d6976)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:08.160439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:08.162016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 580/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 07:48:08 2022] GET /identity/v3/users/be00e8c424a0454d91354428fd7d6976 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:08.167736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:08.189927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:08.264095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1716d1-0822-42f3-a462-12ed1ffd6cb2 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:08.295834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:08.296425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:08.296702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:08.296972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:08.360764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:08.362774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:08.383489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:08.458852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 618/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:48:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:48:08.464977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:08.561967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64947b8d-a71d-43fb-bc31-6dd70f466661 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:08.571795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:08.594952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:08.596149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:08.596421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:08.596688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] PATH_INFO: `/v3/projects/2795fcb5e37c4ba6a636229feebd5304` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:08.597801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=2795fcb5e37c4ba6a636229feebd5304)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:08.599222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d6b5b-7ef3-419b-bb4a-f2a7c0932fd5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:08.600805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 581/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 07:48:08 2022] GET /identity/v3/projects/2795fcb5e37c4ba6a636229feebd5304 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:08.606820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:48:08.702219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45a48f-6cf4-46ba-86e6-2d878b28e900 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:48:08.723445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:08.738981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:48:08.739590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:48:08.739868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:48:08.740138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:48:08.740857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2795fcb5e37c4ba6a636229feebd5304, user.id=be00e8c424a0454d91354428fd7d6976) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:48:08.741165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:48:08.742639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249fd22-a72f-4eba-afcc-29428e53bcbd None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:48:08.748215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 619/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:48:08 2022] GET /identity/v3/role_assignments?user.id=be00e8c424a0454d91354428fd7d6976&scope.project.id=2795fcb5e37c4ba6a636229feebd5304&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:48:18.591202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=11 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:48:18.746070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=8 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:04.579510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882a6b-976e-4c31-833e-fc2ae624aae5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:04.580157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882a6b-976e-4c31-833e-fc2ae624aae5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:04.580157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882a6b-976e-4c31-833e-fc2ae624aae5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:04.581387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 582/1201] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:04.587526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:04.588099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:04.588199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:04.613929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:04.615281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:04.707924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 620/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:49:04.991825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1882a6b-976e-4c31-833e-fc2ae624aae5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:05.128868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1882a6b-976e-4c31-833e-fc2ae624aae5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:05.173672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:05.174698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:05.175171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:05.175670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:05.194163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:05.195782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:05.212962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:05.214117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:05.215106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:05.215106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:05.215106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:05.215106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 583/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:49:05.223173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:05.359218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23215172-ae3f-4289-a8c1-f6a1c588657c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:05.408255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:05.415441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:05.416101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:05.416398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:05.416668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:05.417338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:05.417645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:05.422353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d64e4-6bde-4301-850f-fe9f21bed2bb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:05.435169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 621/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:05.441382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:05.508451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:05.577929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced6897-b547-4269-be7e-b2672e13894d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:05.626705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:05.627515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:05.627651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:05.627874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:05.628572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:05.630203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a3d75-27ff-46fa-86bc-ca5e83136d0f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:05.674007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 584/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 07:49:05 2022] POST /identity/v3/users => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:49:08.312323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35295c50-8c33-4e8c-b4c3-a65895eadf2e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:08.312891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35295c50-8c33-4e8c-b4c3-a65895eadf2e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:08.312891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35295c50-8c33-4e8c-b4c3-a65895eadf2e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:08.314435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 622/1206] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:49:08.319998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:08.320421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:08.320530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:08.340715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:08.342129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:08.429308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 585/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:08.730537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35295c50-8c33-4e8c-b4c3-a65895eadf2e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:08.856596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35295c50-8c33-4e8c-b4c3-a65895eadf2e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:08.900812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:08.902083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:08.902083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:08.902083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:08.919040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:08.924965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:08.926042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:08.926999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 623/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:49:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:49:08.935040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:08.986143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:09.054807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd557f-9ac3-4f26-a0a4-82aa6a02991d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:09.096230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:09.096874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:09.097156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:09.097444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:09.098179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:09.098489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:09.099979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:09.110717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 586/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:49:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:09.116973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:09.174581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:09.237855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81981fae-611f-4890-851c-6ddd66a0b105 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:09.277264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:09.277944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:09.278227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:09.278499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:09.293645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:09.295490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:09.312246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:09.313724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:09.314667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:09.314667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:09.314667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:09.314667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 624/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:09.319227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:09.430220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:09.434028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d543-ee42-446d-ac61-d744b3520395 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:09.473349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:09.474375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:09.474903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:09.475381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:09.476320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:09.476886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:09.478513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:09.493815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 587/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:09.500520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:09.625995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0f6df-4339-48c3-9d5d-9b71fe0f0bfa None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:09.628407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:09.663284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:09.663918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:09.664198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:09.664467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:09.673881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:09.675363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:09.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:09.688589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:09.689546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 625/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:09.694443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:09.803425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbfac-a44b-4b55-89fe-dd903b8d846d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:09.844900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:09.845680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:09.846080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:09.846445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:09.847282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:09.847687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:09.849228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:09.871744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 588/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:49:09.878483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:10.005488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0eca6-cfc8-4d57-82b4-d7b8fb8f75fc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:10.053254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:10.054171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:10.054535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:10.054962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:10.091654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:10.135824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:10.137373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:10.251060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 626/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:49:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 07:49:10.257867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:10.280628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:10.364376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84d546-075c-4a3f-9f88-84ccb90a643d None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:10.400895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:10.401528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:10.401836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:10.402109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:10.412854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:10.414320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:10.431008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:10.432933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:10.434162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:10.434162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:10.434162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:10.434162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 589/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:10.438459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:10.503922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:10.542690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c1749-84f0-4466-9289-0a1e54a39d1a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:10.585839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:10.585839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:10.585839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:10.585839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:10.586415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:10.586415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:10.586668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:10.604845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 627/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:10.606278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:10.687113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:10.720541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6421-9863-41e2-b7cc-54d1444e9f4f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:10.755475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:10.756184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:10.756463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:10.756735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:10.763486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:10.765492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:10.777702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:10.778834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:10.780410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 590/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:10.785521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:10.895330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7dd6-5e69-4a78-b0eb-bddb827689fb None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:10.920128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:10.933077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:10.933683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:10.933999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:10.934270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:10.934950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:10.935251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:10.936737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6fef2-d19e-41ed-91df-44d97261ea5d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:10.955269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 628/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:10.956386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:11.066962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d3933-9fdd-451a-8801-39f8cda0847e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:11.097749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:11.104867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:11.105685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:11.106072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:11.106816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:11.107914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=e6fd2b2ee55f419195b935ba13743dc5) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:11.107914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:11.109518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5cb98-a16a-49f6-b10a-f0851a2513aa None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:11.116018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 591/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:49:10 2022] GET /identity/v3/role_assignments?user.id=e6fd2b2ee55f419195b935ba13743dc5&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:49:13.372772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a960178-4d00-4d96-bda1-6db463db4518 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:13.373878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a960178-4d00-4d96-bda1-6db463db4518 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:13.373878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a960178-4d00-4d96-bda1-6db463db4518 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:13.375215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 629/1220] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:13.380442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:13.380877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:13.381143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:13.401321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:13.402791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:13.477804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 592/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:13.770159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a960178-4d00-4d96-bda1-6db463db4518 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:13.886184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a960178-4d00-4d96-bda1-6db463db4518 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:13.930392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:13.931022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:13.931339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:13.931609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:13.943546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:13.948396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:13.953793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:13.954929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 630/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:49:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:13.955788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.064086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8ef2e-5f7b-4979-a718-ea4c4a99aa12 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.087810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:14.102268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.102932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.103150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.103423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.104158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:14.104469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.105842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:14.117399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 593/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:49:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:14.123609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.172668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:14.235586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18569d4-0b58-427a-93d3-ff1b3f074e11 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.273314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.274035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.274258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.274523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.285849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.287498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:14.304119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:14.310747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:14.312146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:14.312146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:14.312146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:14.312146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 631/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:14.312397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.421841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a39ab-d234-40ed-89af-97d74c8b5eb2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.459246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.459855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.460131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.460399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.461035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:14.461332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.462801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:14.477765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 594/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:14.482554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.588987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:14.596829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f30cb8-e45e-4154-ba40-fecf71b92dd0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.630343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.630965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.631240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.631508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.639334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.640788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:14.651634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:14.658027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:14.659236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 632/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:14.659935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.708911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:14.762358 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a08816-7552-46d5-bd05-18c434cd88bd None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.797295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.797947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.798224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.798492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.799211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:14.799512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.800930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:14.819740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 595/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:14.825602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:14.939898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc23b83-634e-45f7-931a-a1f1eba21477 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:14.974497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:14.975047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:14.975321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:14.975589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:14.989644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:14.991160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:15.005689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:15.006726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:15.007618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:15.007618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:15.007618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:15.007618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 633/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:15.012695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:15.115194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6faa-871f-4f7e-ad57-b3a480098cd5 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:15.117527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:15.151237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:15.151837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:15.152119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:15.152374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:15.153003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:15.153295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:15.154688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:15.166586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 596/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:15.173229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:15.183962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:15.280266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502f3d-c593-4449-a9bb-9b5359569ce0 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:15.320544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:15.321340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:15.321651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:15.321967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:15.330086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:15.331522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:15.345610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:15.346557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:15.347455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 634/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:15.353535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:15.462054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d47a64-7f8b-4cec-b742-20e37e926b75 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:15.500246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:15.500859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:15.501128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:15.501398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:15.502128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:15.502428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:15.503847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ecb8e-e879-4690-92f8-38577a3ca367 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:15.517261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 597/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:15.526669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:15.600517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:15.640172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722896b-adfd-4722-b573-cd8447a9e2fa None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:15.675249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:15.675903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:15.676179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:15.676451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/e6fd2b2ee55f419195b935ba13743dc5/roles/e26f18bbf9d24836837e34167a16a9b6` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:15.691368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=e6fd2b2ee55f419195b935ba13743dc5, role_id=e26f18bbf9d24836837e34167a16a9b6)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:15.692799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966690d-c4d5-4808-b4a8-eac942257932 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:15.705718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 635/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:49:15 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/e6fd2b2ee55f419195b935ba13743dc5/roles/e26f18bbf9d24836837e34167a16a9b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:49:17.842780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9f3c-9049-43ba-a7a1-17fbab4c2fa2 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:17.843332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9f3c-9049-43ba-a7a1-17fbab4c2fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:17.843332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d9f3c-9049-43ba-a7a1-17fbab4c2fa2 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:17.844667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 598/1233] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:17.850030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:17.850466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:17.850606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:17.868800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:17.870014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:17.941425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 636/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:18.228772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d9f3c-9049-43ba-a7a1-17fbab4c2fa2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:18.255499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:18.346617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d9f3c-9049-43ba-a7a1-17fbab4c2fa2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:18.386204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:18.386927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:18.387218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:18.387587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:18.398467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:18.403585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:18.404677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:18.405543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 599/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 07:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:18.411113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:18.465307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:18.523502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421365cb-51d3-41a0-8ed8-8d37eb6bb2f1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:18.560187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:18.560836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:18.561199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:18.561484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:18.562211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:18.562637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:18.564216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:18.575827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 637/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 07:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:18.583156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:18.697052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:18.713651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3da3f-0c47-4471-b67e-f7cbcd58f385 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:18.755478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:18.756162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:18.756450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:18.756826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:18.769656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:18.771456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:18.789758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:18.793845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:18.795443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:18.795443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:18.795443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:18.795443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 600/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:18.797253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:18.917665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771b977-c0ba-474b-887f-dea8f68177a9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:18.928338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:18.955778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:18.956393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:18.956703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:18.956947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:18.957595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:18.957925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:18.959330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:18.971691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 638/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:18.977588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:19.093235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2fe6b-56cd-41ac-bccb-f76b77df2892 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:19.131755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:19.132347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:19.132661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:19.132930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:19.135714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:19.142725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:19.144272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:19.158031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:19.159602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:19.162183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 601/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:49:19.165336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:19.286986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa65c1-01db-41a7-83fc-1aed2b96a448 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:19.323491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:19.324105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:19.324374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:19.324712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:19.325590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:19.326505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:19.327945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:19.347271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 639/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:19.352987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:19.462935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aec82f-5571-49c7-a324-20259bf62c01 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:19.500178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:19.500926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:19.501350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:19.501561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:19.591536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:19.599514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:19.601006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:19.713054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 602/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:49:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:49:19.719132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:19.749578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:19.831461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de6582-8bc5-4d9c-b40a-014f598e2fa6 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:19.873827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:19.874502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:19.874840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:19.875361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:19.887750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:19.889234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:19.905160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:19.910429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:19.911479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:19.911479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:19.911479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:19.911479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 640/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:49:19.912105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:20.021831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:20.030625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33e397-e43a-4113-8fa4-a2ee3d2373d4 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:20.068772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:20.069410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:20.069693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:20.070079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:20.070863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:20.071194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:20.072680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:20.085170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 603/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:20.091000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:20.183637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:20.208670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784edeb-5861-4ea8-a2c8-c203bc057069 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:20.247663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:20.248730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:20.249192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:20.249655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:20.257294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:20.258947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:20.271413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:20.272429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:20.273491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:20.275143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 641/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 07:49:20.281915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:20.392193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31745f2-ef73-4c9a-bd91-35cc747df31e None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:20.431342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:20.431956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:20.432596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:20.432915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:20.433614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:20.434084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:20.436218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f5553-e12e-4ab3-b463-b8b50c0d9c01 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:20.450973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 604/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:20.457504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:20.571317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545c7f4-0ef0-44ef-9154-2c386fedf888 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:20.616167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:20.616807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:20.617085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:20.617356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:20.618239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] RBAC: Adding query filter params (role.id=e26f18bbf9d24836837e34167a16a9b6, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=e6fd2b2ee55f419195b935ba13743dc5) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:20.618556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:20.620142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564fce-1af2-457c-9eee-fb33c69757bf None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:20.628956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 642/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:49:20 2022] GET /identity/v3/role_assignments?user.id=e6fd2b2ee55f419195b935ba13743dc5&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=e26f18bbf9d24836837e34167a16a9b6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:23.120026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae29ae-4393-4bf1-abc2-ef01c9d181a1 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:23.120577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae29ae-4393-4bf1-abc2-ef01c9d181a1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:23.120577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae29ae-4393-4bf1-abc2-ef01c9d181a1 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:23.121939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 605/1247] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:23.130883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:23.130883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:23.130883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:23.135349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:23.148277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:23.149480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:23.231015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 643/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:23.524673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae29ae-4393-4bf1-abc2-ef01c9d181a1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:23.545118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:23.654347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae29ae-4393-4bf1-abc2-ef01c9d181a1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:23.690968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:23.691620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:23.691893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:23.692161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:23.704016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:23.708828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:23.709815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:23.710702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 606/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:23.716114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:23.818982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb0ceb-3282-4bc8-9d8e-ec764b350c93 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:23.848728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:23.854539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:23.855161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:23.855436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:23.855706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:23.856403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:23.856702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:23.858185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:23.867207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 644/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:23.873645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:23.961021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:23.984652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303e62b-5920-43a4-87db-9ceb928edc94 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.026219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.026848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.027151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.027419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:24.037558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:24.039012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:24.053250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:24.054396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:24.055488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:24.055488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:24.055488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:24.055488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 607/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:24.062417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:24.165900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b876-ed14-4fe2-ac68-f8755d073c75 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.200609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.201214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.201548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.201856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:24.202519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:24.202817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:24.204210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:24.216177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 645/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:24.222065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:24.324240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ff2-40ee-465d-822f-21d019d09351 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.351218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:24.359633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.360264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.360546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.360821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:24.368127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:24.369559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:24.381117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:24.382409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:24.383800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 608/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:24.388117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:24.442494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:24.485539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c543e9-ce44-4e04-ac24-a4eb3e9c1d64 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.521591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.522229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.522511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.522784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:24.523520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:24.523827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:24.525271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:24.543542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 646/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:24.549996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:24.656866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32ae88-18cb-41d3-9165-0daaf40d1769 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.689700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.690338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.690621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.690897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:24.745089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:24.757406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:24.758907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:24.853420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 609/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:49:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:49:24.859809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:24.961012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27e4ab-a521-443b-952b-1a3476044255 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:24.974240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:24.994789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:24.995458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:24.995688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:24.995950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:25.005543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:25.006996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:25.021210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:25.022366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:25.023318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:25.023318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:25.023318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:25.023318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 647/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:25.027984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:25.112360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:25.124204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ae8a1-614f-45f9-9d17-62b680336123 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:25.156995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:25.157585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:25.157889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:25.158156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:25.158779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:25.159075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:25.160412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:25.171086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 610/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:25.176447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:25.271584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6b7-25bc-4f6c-9844-0a8dde5e59bc None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:25.303296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:25.303914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:25.304192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:25.304459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:25.311639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:25.313055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:25.323889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:25.324970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:25.326844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 648/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:25.331343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:25.428457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a76508-6d89-42f3-bf8f-301bf08c13b2 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:25.461578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:25.462207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:25.462487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:25.462759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:25.463439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:25.463741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:25.465179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9e612-a099-4365-ad97-c293bf4849c4 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:25.476650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 611/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:25.482637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:25.507627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:25.582925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ebdcb-19dc-4042-a2c7-38f30ffc3a83 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:25.619507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:25.620101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:25.620376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:25.620645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:25.621360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=e6fd2b2ee55f419195b935ba13743dc5) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:25.621664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:25.623625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b102e-e7ee-47f1-abfb-8580ceb70c7c None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:25.628787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 649/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:49:25 2022] GET /identity/v3/role_assignments?user.id=e6fd2b2ee55f419195b935ba13743dc5&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:27.737806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2034a3-f0fb-4210-a66e-5404955d1a9b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:27.738363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2034a3-f0fb-4210-a66e-5404955d1a9b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:27.738363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2034a3-f0fb-4210-a66e-5404955d1a9b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:27.739668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 612/1261] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:27.744672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:27.745092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:27.745217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:27.762890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:27.764045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:27.828162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 650/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 07:49:28.104315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2034a3-f0fb-4210-a66e-5404955d1a9b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.115070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:28.202827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2034a3-f0fb-4210-a66e-5404955d1a9b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:28.235765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:28.236384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:28.236662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:28.236934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:28.246783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:28.252036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:28.253224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:28.254293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 613/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:28.259288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.277481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:28.355631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce2065-bd37-462f-b87d-c1510a2ed9ef None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:28.390862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:28.391461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:28.391739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:28.392013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:28.392662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:28.392975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:28.394353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:28.402050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 651/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:28.407751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.472406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:28.504542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450673-437f-4072-8b6e-a9a385331e2f None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:28.538064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:28.538674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:28.538948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:28.539216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:28.548833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:28.550290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:28.565365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:28.566442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:28.567688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:28.567688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:28.567688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:28.567688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 614/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:28.572924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.671620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:28.691441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec767b8-5c9a-44e3-9ba7-0e49201d6fb8 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:28.723373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:28.723971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:28.724248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:28.724517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:28.725149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:28.725451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:28.726846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:28.738628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 652/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:28.744368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.841069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b39381-ece3-487c-9f72-ce74b77c220c None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:28.860104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:28.874487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:28.875102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:28.875381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:28.875651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:28.882809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:28.884246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:28.895441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:28.896479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:28.897469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 615/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:28.902439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:28.996877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147779c-eb47-4697-8f69-644ac00773ef None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.029262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.029874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.030147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.030411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.031117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:29.031412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.032848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.041121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:29.050056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 653/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:29.055633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:29.158537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc252b-1111-4303-96ef-23f8fb0b68c3 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.190849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.191459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.191733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.191997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.202020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.203441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:29.217482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:29.218473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:29.219617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:29.219617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:29.219617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:29.219617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 616/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:29.224289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:29.321981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60210233-b236-4911-a0e8-91a63296a948 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.354510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.355111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.355389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.355659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.356326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:29.357192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.358590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.369350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 654/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:29.375062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:29.408401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:29.474740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d838a0-6a1e-42a5-a5a8-a5084a1c5040 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.506265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.506876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.507150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.507420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.514253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.515678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:29.526570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:29.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:29.529187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 617/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:29.534110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:29.595968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:29.638324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7735e77-f1e3-4e08-a2d8-ad80a55678ee None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.670728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.671328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.671609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.671880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.672565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:29.672866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.674330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf10b13-94bc-4b09-adec-66e1c08f0ddc None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.686054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 655/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:29.692081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:29.780854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:29.790043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51614a3-a4f2-496d-9c45-accabaebecfb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:29.822272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:29.822915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:29.823191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:29.823459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] PATH_INFO: `/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/e6fd2b2ee55f419195b935ba13743dc5/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:29.826024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a0991faba72483b98d9ff2d19d1e952, user_id=e6fd2b2ee55f419195b935ba13743dc5, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:29.827433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ebccf-2963-45f1-aaa6-2ae383327a60 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:29.839367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 618/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 07:49:29 2022] PUT /identity/v3/projects/2a0991faba72483b98d9ff2d19d1e952/users/e6fd2b2ee55f419195b935ba13743dc5/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:49:31.936850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414b8e-c704-4fe8-9933-1d64bb9c68c9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:31.937348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414b8e-c704-4fe8-9933-1d64bb9c68c9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:31.937438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414b8e-c704-4fe8-9933-1d64bb9c68c9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:31.938745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 656/1274] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:31.944093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:31.944541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:31.944643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:31.962553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:31.963742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:32.028954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 619/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:32.300613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e414b8e-c704-4fe8-9933-1d64bb9c68c9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:32.398253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e414b8e-c704-4fe8-9933-1d64bb9c68c9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:32.415011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:32.431541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:32.432455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:32.432730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:32.432996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:32.442989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 07:49:32.447696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:32.448794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:32.449645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 657/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 07:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:32.455381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:32.533505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:32.552449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a9678-2871-46ee-a65e-691dc01d1ced None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:32.590495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:32.591822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:32.592123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:32.592437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:32.593229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:32.593536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:32.595163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:32.604558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 620/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 07:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:32.611158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:32.709480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551faef8-8720-4f49-82d4-50665f8ff9db None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:32.741694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:32.742335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:32.742607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:32.742870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:32.752869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:32.754331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:32.768690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:32.769876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:32.770816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:32.770816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:32.770816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:32.770816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 658/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:32.775486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:32.872721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef4594-9e66-4363-8bf9-3896adcf1b1a None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:32.905649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:32.906277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:32.906556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:32.906828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:32.907465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:32.907764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:32.909119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:32.920584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 621/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:32.926156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:32.957251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:33.024911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde80ae-72f1-48d6-891a-e6abd011db07 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.056619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:33.057497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:33.057801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:33.058074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:33.065648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:33.067107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 07:49:33.078435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:33.079645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:33.080674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 659/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:33.085095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:33.088844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:33.182291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277378d8-ab38-46c1-b902-62ce659de4e1 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.214905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:33.215496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:33.215795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:33.216062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:33.216778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:33.217078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:33.218517 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:33.235264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 622/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 07:49:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:33.240811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:33.328351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:33.338064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196f09-071a-4ce1-aaf1-d42fd77a4e16 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.370186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:33.370778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:33.371055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:33.371332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:33.436988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:33.438404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:33.523520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:33.534182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 660/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 07:49:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 07:49:33.539954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:33.600952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:33.639103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e328-5846-49a7-9f4a-96963dde4bfc None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.690873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:33.691421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:33.691694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:33.691960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:33.701865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:33.703295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 07:49:33.717866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:33.718958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 07:49:33.720225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 07:49:33.720225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 07:49:33.720225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:33.720225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 623/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:33.724636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:33.820745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239f98-07a8-4940-b42c-f2d8298c4ab1 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.853385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:33.854010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:33.854287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:33.854556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:33.855184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:33.855481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:33.857414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:33.868110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 661/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 07:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:33.873551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:33.971142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450316e-3836-4f9f-9c69-946e6f845bbb None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:33.983979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:34.004136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:34.004764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:34.005040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:34.005311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:34.013001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:34.014488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 07:49:34.026198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:34.027276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: ERROR keystone.server.flask.application Sep 13 07:49:34.030106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 624/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 07:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:34.033925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:34.087717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:34.136594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac339f-2ea1-4f7d-95f8-80dc132eab1a None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:34.169357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:34.169983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:34.170264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:34.170536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:34.171219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:34.171523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:34.172946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88abb74-5dfb-49f3-9d7e-c58bf706e2d8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:34.184655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 662/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 07:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:34.190660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:34.287224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60530c76-ae2e-43df-9777-e9d325903f04 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:34.321266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:34.321894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:34.322173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:34.322444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:34.323167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] RBAC: Adding query filter params (role.id=7b12b864f5fc46169140cae8eff50f57, scope.project.id=2a0991faba72483b98d9ff2d19d1e952, user.id=e6fd2b2ee55f419195b935ba13743dc5) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:34.323473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:34.324952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6a09e-a82a-43bd-81fd-7783a5833047 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:34.331449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 625/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 07:49:34 2022] GET /identity/v3/role_assignments?user.id=e6fd2b2ee55f419195b935ba13743dc5&scope.project.id=2a0991faba72483b98d9ff2d19d1e952&role.id=7b12b864f5fc46169140cae8eff50f57 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:36.493049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f97ff0-d942-45b6-842e-5d992c1f0f53 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:36.493581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f97ff0-d942-45b6-842e-5d992c1f0f53 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:36.493581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f97ff0-d942-45b6-842e-5d992c1f0f53 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:36.494929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 663/1288] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:36.500845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:36.501270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:36.501403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:36.519307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:36.520484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:36.549608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:36.592772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 626/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:36.869923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f97ff0-d942-45b6-842e-5d992c1f0f53 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:36.970233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f97ff0-d942-45b6-842e-5d992c1f0f53 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:36.980473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:37.002983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:37.003600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:37.003873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:37.004138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:37.004707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:37.006086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:37.014947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:37.016218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:37.017093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 664/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:49:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:37.022125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:37.118209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2827e53-1f2a-40b0-bda4-2414e662abbf None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:37.151095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:37.151632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:37.151911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:37.152184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:37.152829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:37.153695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:37.155097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fde7c-e616-4601-9725-f9b600f0979a None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:37.159710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 627/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 07:49:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:37.165169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:37.263938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aad08-40ac-4839-a292-dea084f9cdf9 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:37.295498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:37.296095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:37.296368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:37.296634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] PATH_INFO: `/v3/services` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:37.297251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:37.297547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:37.298939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0401ab-362b-434b-b284-33499d7db9f8 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:37.303138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 665/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 07:49:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:39.423222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0183d-6022-4209-b462-87e7b13e75c7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:39.423785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0183d-6022-4209-b462-87e7b13e75c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:39.423785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0183d-6022-4209-b462-87e7b13e75c7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:39.425091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 628/1293] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:39.430343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06e86-0ad9-4d04-9d90-59b7967d81d1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:39.430770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06e86-0ad9-4d04-9d90-59b7967d81d1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:39.430882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06e86-0ad9-4d04-9d90-59b7967d81d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:39.449676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-81b06e86-0ad9-4d04-9d90-59b7967d81d1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:39.451049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-81b06e86-0ad9-4d04-9d90-59b7967d81d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:39.479604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:39.516920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 666/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:39.790484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0183d-6022-4209-b462-87e7b13e75c7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:39.821251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:39.890002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0183d-6022-4209-b462-87e7b13e75c7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:39.926008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:39.926622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:39.926892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:39.927160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:39.927807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:39.929362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f21b00-2bea-4271-bd31-0b5543fa0021 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:39.944312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 629/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 07:49:39 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:49:42.074546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da729e-26de-4209-9dcf-013dd39c2056 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:42.075098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da729e-26de-4209-9dcf-013dd39c2056 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:42.075098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da729e-26de-4209-9dcf-013dd39c2056 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:42.076405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 667/1296] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:42.081457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:42.081910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:42.082033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:42.100156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:42.101321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:42.171591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 630/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:42.441573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da729e-26de-4209-9dcf-013dd39c2056 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:42.538848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da729e-26de-4209-9dcf-013dd39c2056 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:42.575499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:42.576229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:42.576522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:42.576787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:42.577549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:42.578918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:42.588731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:42.590249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:42.591151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 668/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:49:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:42.596459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:42.617749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:42.700808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08260b-a99f-4293-9a68-27e686adc68c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:42.732816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:42.733420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:42.733698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:42.734020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:42.734654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:42.734955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:42.736328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:42.741261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 631/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 07:49:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:49:42.747791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:42.778391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:42.845008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77265dd0-c659-41d0-8109-8a4777efe90d None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:42.877095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:42.877681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:42.877989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:42.878261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:42.878980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] RBAC: Adding query filter params (interface=public, service_id=f40517be82654251a336598e47ec9a25, region_id=RegionOne) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:42.879284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:42.880645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e628c-4fc5-45da-8128-dda644247a7e None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:42.885557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 669/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 07:49:42 2022] GET /identity/v3/endpoints?service_id=f40517be82654251a336598e47ec9a25&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:42.891693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:42.970672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:42.988965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63079485-c664-4624-974d-c6600bd85f00 None admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:43.021709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:43.022331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:43.022604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:43.022873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:43.023467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:43.024811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f26b28-411f-44ea-9bf7-c262389bea78 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:43.031435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 632/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 07:49:42 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:49:45.136161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f26-0ed8-4c9a-9ddd-acd3d00c63d4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:45.136716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f26-0ed8-4c9a-9ddd-acd3d00c63d4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:45.136716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1f26-0ed8-4c9a-9ddd-acd3d00c63d4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:45.138623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 670/1302] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:49:45.143614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:45.143984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:45.144159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:45.162366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:49:45.163526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:49:45.226864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 633/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:49:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 07:49:45.491903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1f26-0ed8-4c9a-9ddd-acd3d00c63d4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:45.584332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:45.591476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1f26-0ed8-4c9a-9ddd-acd3d00c63d4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:45.623569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:45.624179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:45.624453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:45.624718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:45.625286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:45.626677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 07:49:45.635057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 07:49:45.636206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application with Lock( Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self._enter() Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application return self.creator() Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application created_value = creator( Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: ERROR keystone.server.flask.application Sep 13 07:49:45.638142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 671/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 07:49:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 07:49:45.643391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:45.716617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:45.740303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cfae-f12f-4747-9d5e-9b0ec37bdca0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:45.773404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:45.773965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:45.774238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:45.774502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] PATH_INFO: `/v3/services` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:45.775120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:49:45.775416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:45.776763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340c0ac-5092-429f-9c6b-333c184ab7f8 None admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:45.781756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 634/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 07:49:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:49:45.787510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:49:45.882714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b46ee-b7e4-4a76-aee1-9c7ab6894b09 None admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:49:45.915649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:49:45.916243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:49:45.916518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:49:45.916783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:49:45.917436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:49:45.919335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c56d7-7d41-4577-9476-505b20fa3191 None admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:49:45.938590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 672/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:49:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:49:55.779366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=30 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:49:55.934533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:50.044909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ecca-7559-46ff-a45d-8c4b885e3ce6 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:50.045408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ecca-7559-46ff-a45d-8c4b885e3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:50.045513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ecca-7559-46ff-a45d-8c4b885e3ce6 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:50.052131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e46398-07d4-455d-8970-be792a9d3d84 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:50.053082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e46398-07d4-455d-8970-be792a9d3d84 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:50.053658 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e46398-07d4-455d-8970-be792a9d3d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:50.056265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 635/1307] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:50:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:50:50.183847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f5e46398-07d4-455d-8970-be792a9d3d84 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:50:50.185207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f5e46398-07d4-455d-8970-be792a9d3d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:50.301778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 673/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:50:50.603785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:50.604081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:50.604329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:50.605766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 636/1309] 10.222.0.15 () {54 vars in 965 bytes} [Tue Sep 13 07:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:50:50.611824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592687b-ed8e-4b67-b720-3fb56eaf4915 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:50.612144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592687b-ed8e-4b67-b720-3fb56eaf4915 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:50.612401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592687b-ed8e-4b67-b720-3fb56eaf4915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:50.697628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f592687b-ed8e-4b67-b720-3fb56eaf4915 None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:50:50.698993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f592687b-ed8e-4b67-b720-3fb56eaf4915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:50.721408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:50.781282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 674/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Tue Sep 13 07:50:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:50:50.790540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:50:50.792399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:50.925431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14278895-c640-4304-aca6-922ae6b1f363 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:50:51.014940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:50:51.015594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:51.015870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:51.016139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:51.017720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:51.080162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:51.145134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:50:51.146847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087210d-d7d3-44cd-a5a2-b784bcc0d3f9 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:50:51.173771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 637/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:50:53.700585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5d88b-6559-41ec-82b1-4767f922860d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:53.701092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5d88b-6559-41ec-82b1-4767f922860d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:53.701092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5d88b-6559-41ec-82b1-4767f922860d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:53.702401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 675/1312] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:50:53.707699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a1ea1-d137-45e9-a3e4-f59e15d8b0ec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:53.708103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a1ea1-d137-45e9-a3e4-f59e15d8b0ec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:53.708245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a1ea1-d137-45e9-a3e4-f59e15d8b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:53.778278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7f3a1ea1-d137-45e9-a3e4-f59e15d8b0ec None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:50:53.779513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7f3a1ea1-d137-45e9-a3e4-f59e15d8b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:53.874314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 638/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:50:54.171556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:54.171833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:54.172103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:54.173425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 676/1314] 10.222.0.15 () {54 vars in 965 bytes} [Tue Sep 13 07:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:50:54.179451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3e225-b4d1-49e5-91c7-5c8de94d534b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:54.179731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3e225-b4d1-49e5-91c7-5c8de94d534b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:54.180001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3e225-b4d1-49e5-91c7-5c8de94d534b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:54.192808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:54.256849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-eee3e225-b4d1-49e5-91c7-5c8de94d534b None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:50:54.258072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eee3e225-b4d1-49e5-91c7-5c8de94d534b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:54.331537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 639/1315] 10.222.0.15 () {58 vars in 1062 bytes} [Tue Sep 13 07:50:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:50:54.340540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:50:54.342094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:54.443647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8d965-a5e9-42db-8753-8a124f1fdf83 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:50:54.527828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:54.530666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:50:54.531478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:54.531865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:54.532321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:54.534244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:54.639191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:50:54.640609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb711c63-5279-4966-92cb-987bcd8e0908 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:50:54.658324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 677/1316] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:50:57.317488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:57.318103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:57.318103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:57.319412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 640/1317] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:50:57.324715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553c462-f7be-455e-9832-d5d477b68cc6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:57.325120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553c462-f7be-455e-9832-d5d477b68cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:57.325255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553c462-f7be-455e-9832-d5d477b68cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:57.389028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e553c462-f7be-455e-9832-d5d477b68cc6 None None] MFA Rules not processed for user `7c91ef32acfb46029d0aa8660b4c5b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:50:57.390234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e553c462-f7be-455e-9832-d5d477b68cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:57.441533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:57.485291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 678/1318] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:50:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:50:57.770559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:50:57.772068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:57.872481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf0526-7a45-493a-80f5-df4c582b0cf5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:50:57.875241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:50:57.948220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:50:57.948827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:50:57.949105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:50:57.949376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:50:57.951007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:50:58.058153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:50:58.059619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b9109-273f-4374-aa20-d6fb425463b0 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:50:58.078022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 641/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:50:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:00.690194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:00.690761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:00.690761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:00.692065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 679/1320] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:00.697841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b375-a511-4582-90bc-b8d6e11eccd7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:00.698237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b375-a511-4582-90bc-b8d6e11eccd7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:00.698395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b375-a511-4582-90bc-b8d6e11eccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:00.737390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:00.764449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7586b375-a511-4582-90bc-b8d6e11eccd7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:00.765644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7586b375-a511-4582-90bc-b8d6e11eccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:00.860010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 642/1321] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:01.142082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:01.143630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:01.246369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f9a62-7ceb-442d-b464-ce4383b402b6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:01.318842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:01.319442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:01.319718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:01.319987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:01.321588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:01.328259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:01.428115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:01.429552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c22f-ff5e-485b-a9b6-29e98685803a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:01.447615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 680/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:04.231096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:04.231629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:04.231629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:04.233778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 643/1323] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:04.238853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd41259-1749-454b-a425-b91fd7103f30 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:04.239393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd41259-1749-454b-a425-b91fd7103f30 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:04.239393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd41259-1749-454b-a425-b91fd7103f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:04.312535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4bd41259-1749-454b-a425-b91fd7103f30 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:04.313784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4bd41259-1749-454b-a425-b91fd7103f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:04.374496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:04.416878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 681/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:04.697819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:04.699344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:04.802147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b873478-03c9-41a5-9708-c18dedb1134f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:04.817924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:04.881566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:04.883157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:04.883625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:04.884056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:04.885701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:04.995381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:04.996854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a1d5b-96c0-413c-82a3-b8c8aad38333 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:05.015408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 644/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:07.412309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:07.412682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:07.412682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:07.414820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 682/1326] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:07.420027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85de14-e0ec-4ddb-8761-b5c4b1eafa2a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:07.420433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85de14-e0ec-4ddb-8761-b5c4b1eafa2a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:07.420588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85de14-e0ec-4ddb-8761-b5c4b1eafa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:07.468196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:07.487560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ed85de14-e0ec-4ddb-8761-b5c4b1eafa2a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:07.488774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed85de14-e0ec-4ddb-8761-b5c4b1eafa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:07.586688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 645/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:07.872726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:07.874370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:07.981703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4856e-873a-4518-98d1-979bb986e2d8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:08.056913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:08.057545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:08.057836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:08.058112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:08.059698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:08.081282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:08.174827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:08.176204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ad750-cfb0-4aef-bfaf-0be73389e453 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:08.194644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 683/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:10.416266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:10.416813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:10.416813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:10.418274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 646/1329] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:10.423526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7cb56-e95d-4a2a-8728-87ab4aa9c5fe None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:10.423944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7cb56-e95d-4a2a-8728-87ab4aa9c5fe None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:10.424056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7cb56-e95d-4a2a-8728-87ab4aa9c5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:10.489380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-13a7cb56-e95d-4a2a-8728-87ab4aa9c5fe None None] MFA Rules not processed for user `7c91ef32acfb46029d0aa8660b4c5b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:10.490607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-13a7cb56-e95d-4a2a-8728-87ab4aa9c5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:10.560084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:10.593811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 684/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:10.870799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:10.872400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:10.988081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57b9cd-0f9b-4f3f-90d9-4f0d1530563f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:11.010187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:11.080904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:11.081525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:11.081840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:11.082116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:11.083719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:11.191229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:11.192595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d84fbb-7471-4c12-b4df-581f8859a6ca service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:11.212149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 647/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:13.776325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:13.776827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:13.776827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:13.778160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 685/1332] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:13.783466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3b0d-98db-450a-8d82-b5988cf386fd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:13.783875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3b0d-98db-450a-8d82-b5988cf386fd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:13.784023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f3b0d-98db-450a-8d82-b5988cf386fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:13.847164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:13.850139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8b3f3b0d-98db-450a-8d82-b5988cf386fd None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:13.851907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8b3f3b0d-98db-450a-8d82-b5988cf386fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:13.948793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 648/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:14.229285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:14.230850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:14.331127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943bbb6-90b3-4342-ad22-3fd59442557c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:14.402572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:14.403177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:14.403468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:14.403734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:14.405293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:14.432457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:14.511980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:14.513325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61186ccf-649e-4109-a67b-7394e923836f service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:14.530482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 686/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:16.730960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:16.731554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:16.731554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:16.733238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 649/1335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:16.739315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8df9-0311-48cc-a924-d466d1478e28 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:16.739775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8df9-0311-48cc-a924-d466d1478e28 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:16.739895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf8df9-0311-48cc-a924-d466d1478e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:16.804172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-58cf8df9-0311-48cc-a924-d466d1478e28 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:16.805620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-58cf8df9-0311-48cc-a924-d466d1478e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:16.877857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:16.900918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 687/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:17.188124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:17.189685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:17.291159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3e540-330d-403a-9b70-e662c7b59546 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:17.319123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:17.363011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:17.363604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:17.363877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:17.364141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:17.365700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:17.473510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:17.474906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d4891-deb4-4d78-8e9d-e71ed2c5934a service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:17.493494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 650/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:20.030346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:20.030896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:20.030896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:20.032213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 688/1338] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:20.037272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ab744-388d-4be2-8eca-a301f31449ea None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:20.037696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ab744-388d-4be2-8eca-a301f31449ea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:20.037829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ab744-388d-4be2-8eca-a301f31449ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:20.102884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6a8ab744-388d-4be2-8eca-a301f31449ea None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:20.104084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6a8ab744-388d-4be2-8eca-a301f31449ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:20.112197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:20.206776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 651/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:20.485251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:20.486857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:20.593670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc3393-38c8-4977-a6aa-3f6d0e2c7d46 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:20.666583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:20.667184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:20.667459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:20.667725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:20.669333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:20.704851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:20.778734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:20.780121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b9375-8513-4dfd-a69b-2e06018fe93b service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:20.798519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 689/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:24.854591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:24.855169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:24.855169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:24.856491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 652/1341] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:24.861857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4e98-434d-4eb3-9a62-376532c1b5a3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:24.862267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4e98-434d-4eb3-9a62-376532c1b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:24.862408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4e98-434d-4eb3-9a62-376532c1b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:24.932857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c49f4e98-434d-4eb3-9a62-376532c1b5a3 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:24.934031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c49f4e98-434d-4eb3-9a62-376532c1b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:25.021972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:25.033429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 690/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:51:25.330175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:25.331741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:25.438644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eb858-8413-462e-817e-bf70bc34cff9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:25.481520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:25.517434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:25.518069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:25.518346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:25.518610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:25.520171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:25.629852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:25.631209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7cfe8-10d8-4a26-a5ce-5aed15041bb7 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:25.650515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 653/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:28.379753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:28.380355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:28.380355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:28.381678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 691/1344] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:28.386915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af02f-dc71-4812-9b60-54cf8e343f77 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:28.387368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af02f-dc71-4812-9b60-54cf8e343f77 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:28.387468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969af02f-dc71-4812-9b60-54cf8e343f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:28.454281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-969af02f-dc71-4812-9b60-54cf8e343f77 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:28.455480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-969af02f-dc71-4812-9b60-54cf8e343f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:28.473969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:28.557888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 654/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:28.843078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:28.844638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:28.954374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcd08-3daf-479f-9fd4-21e9e2266f33 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:29.035908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:29.036523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:29.036801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:29.037068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:29.039268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:29.082728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:29.151047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:29.152363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490be92-3192-4e49-ac1a-da0b3dc54013 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:29.171205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 692/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:33.619877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:33.620388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:33.620505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:33.621892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 655/1347] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:51:33.627662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c985-cfd4-4c6f-97cf-0df509e94738 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:33.628101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c985-cfd4-4c6f-97cf-0df509e94738 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:33.628331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68c985-cfd4-4c6f-97cf-0df509e94738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:33.704543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-af68c985-cfd4-4c6f-97cf-0df509e94738 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:33.705862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-af68c985-cfd4-4c6f-97cf-0df509e94738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:33.803820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:33.806327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 693/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:34.087650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:34.089238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:34.193710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de507c-fcb2-46ff-bedf-9785f461182e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:34.242108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:34.267338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:34.267935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:34.268205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:34.268477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:34.270078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:34.376433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:34.377810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beaa723-7477-405d-89a1-655b4fefcd66 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:34.395278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 656/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:39.308802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4263b-dfe2-499f-9ea9-4d589c87e7f2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:39.309659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4263b-dfe2-499f-9ea9-4d589c87e7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:39.310121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4263b-dfe2-499f-9ea9-4d589c87e7f2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:39.311877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 694/1350] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:39.316752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255106-9d28-48b1-a544-efa752f31bec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:39.317166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255106-9d28-48b1-a544-efa752f31bec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:39.317298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255106-9d28-48b1-a544-efa752f31bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:39.411996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5d255106-9d28-48b1-a544-efa752f31bec None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:39.413589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5d255106-9d28-48b1-a544-efa752f31bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:39.435019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:39.590900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 657/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:51:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:43.665900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad47a-d669-4309-bee9-4298b472d244 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:43.666456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad47a-d669-4309-bee9-4298b472d244 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:43.666456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcad47a-d669-4309-bee9-4298b472d244 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:43.668067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 695/1352] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:43.673271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85565c-c61f-466f-8489-b3eef3048a2e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:43.673682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85565c-c61f-466f-8489-b3eef3048a2e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:43.673957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85565c-c61f-466f-8489-b3eef3048a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:43.763148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ae85565c-c61f-466f-8489-b3eef3048a2e None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:43.764944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ae85565c-c61f-466f-8489-b3eef3048a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:43.842733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:43.886503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 658/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:51:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:45.367016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6a6a-f88e-4375-a926-528532cbb092 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:45.367572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6a6a-f88e-4375-a926-528532cbb092 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:45.367572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6a6a-f88e-4375-a926-528532cbb092 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:45.368890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 696/1354] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:45.374470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ffeee-aedc-4cbf-bf8c-c093be1d317e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:45.374885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ffeee-aedc-4cbf-bf8c-c093be1d317e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:45.375057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ffeee-aedc-4cbf-bf8c-c093be1d317e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:45.453974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3c5ffeee-aedc-4cbf-bf8c-c093be1d317e None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:45.455195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3c5ffeee-aedc-4cbf-bf8c-c093be1d317e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:45.562626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 659/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:51:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:51:49.121971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf12cd-dc5d-4d93-a538-949847d79901 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:49.122845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf12cd-dc5d-4d93-a538-949847d79901 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:49.123291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf12cd-dc5d-4d93-a538-949847d79901 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:49.128979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 697/1356] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:49.131478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae138e-ae40-47f2-8e1d-42e4bf5ff29f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:49.132586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae138e-ae40-47f2-8e1d-42e4bf5ff29f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:49.133014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae138e-ae40-47f2-8e1d-42e4bf5ff29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:49.175356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:49.210873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3dae138e-ae40-47f2-8e1d-42e4bf5ff29f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:49.212070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3dae138e-ae40-47f2-8e1d-42e4bf5ff29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:49.313304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 660/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:51:49.620322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:49.621001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:49.621630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:49.623169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 698/1358] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 07:51:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:49.631377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89077f29-c456-4b30-98a8-d4797d2b2f68 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:49.631926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89077f29-c456-4b30-98a8-d4797d2b2f68 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:49.632354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89077f29-c456-4b30-98a8-d4797d2b2f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:49.722413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-89077f29-c456-4b30-98a8-d4797d2b2f68 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:49.724377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-89077f29-c456-4b30-98a8-d4797d2b2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:49.779577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:49.866675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 661/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 07:51:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:51:49.874485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:49.876111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:50.029150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6391-944b-477f-a2e6-c10047ba721a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:50.106102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:50.149005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:50.149756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:50.150163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:50.150570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:50.152270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:50.252404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:50.254188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f924-d3e6-4cd3-92f5-af4db4923d1a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:50.273520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 699/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:52.632096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11183-9076-4875-8291-75c73b459668 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:52.632843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11183-9076-4875-8291-75c73b459668 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:52.632843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11183-9076-4875-8291-75c73b459668 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:52.639603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90bac-8077-4112-9312-5e846a370908 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:52.639603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90bac-8077-4112-9312-5e846a370908 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:52.640298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f90bac-8077-4112-9312-5e846a370908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:52.644443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 662/1361] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:52 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:52.713326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f3f90bac-8077-4112-9312-5e846a370908 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:52.714580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3f90bac-8077-4112-9312-5e846a370908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:52.761723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:52.829724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 700/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:53.123344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c11183-9076-4875-8291-75c73b459668 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:53.124881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c3c11183-9076-4875-8291-75c73b459668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:53.297392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c11183-9076-4875-8291-75c73b459668 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:53.310436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:53.447649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:53.448338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:53.448650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:53.448925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:53.450664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:53.544275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:53.545910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbd266-fd65-49f8-9686-7c0b7a7248ef service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:53.564496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 663/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:51:56.079021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab0b9-8ded-407f-9cc8-d1cbdbf165b5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:56.079557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab0b9-8ded-407f-9cc8-d1cbdbf165b5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:56.079557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ab0b9-8ded-407f-9cc8-d1cbdbf165b5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:56.080891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 701/1364] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:56.086370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b75f-90f5-4b6d-8f12-7fc944463a9e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:56.086790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b75f-90f5-4b6d-8f12-7fc944463a9e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:56.086968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b75f-90f5-4b6d-8f12-7fc944463a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:56.102617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:56.162058 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ca63b75f-90f5-4b6d-8f12-7fc944463a9e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:56.163317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca63b75f-90f5-4b6d-8f12-7fc944463a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:56.266062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 664/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:56.562084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:56.562486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:56.562865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:56.564359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 702/1366] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 07:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:56.569988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf083b3-b48d-400e-8da2-0fe589a15112 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:56.570382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf083b3-b48d-400e-8da2-0fe589a15112 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:56.570768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf083b3-b48d-400e-8da2-0fe589a15112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:56.653145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1cf083b3-b48d-400e-8da2-0fe589a15112 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:56.654549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1cf083b3-b48d-400e-8da2-0fe589a15112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:56.691909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:56.785010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 665/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 07:51:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:56.795376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:56.797056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:56.953815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04e41e-6d11-4387-b662-22c2fc6ea601 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:56.978483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:57.082521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:57.090128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:57.090669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:57.091367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:57.093122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:57.192850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:57.194698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d470f7-12e5-4092-902c-368317f73d3e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:57.216193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 703/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:51:57.301952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6fdd-ed4c-4801-a14c-e2def9c61161 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:57.302568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6fdd-ed4c-4801-a14c-e2def9c61161 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:57.302847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef6fdd-ed4c-4801-a14c-e2def9c61161 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:57.304382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 666/1369] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:57.310361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a31a8-0bfb-4856-b778-b276cb80039a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:57.310650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a31a8-0bfb-4856-b778-b276cb80039a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:57.310948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a31a8-0bfb-4856-b778-b276cb80039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:57.349957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:57.394040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-850a31a8-0bfb-4856-b778-b276cb80039a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:57.396206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-850a31a8-0bfb-4856-b778-b276cb80039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:57.517011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 704/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:57.815551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:57.816075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:57.816665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:57.819267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 667/1371] 10.222.0.15 () {54 vars in 964 bytes} [Tue Sep 13 07:51:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:57.826369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292447bd-6888-44ff-8e94-6744766bf026 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:57.826663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292447bd-6888-44ff-8e94-6744766bf026 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:57.826906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292447bd-6888-44ff-8e94-6744766bf026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:57.911922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-292447bd-6888-44ff-8e94-6744766bf026 None None] MFA Rules not processed for user `c8e54e867eee421b8f230c259d3cce68`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:57.913114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-292447bd-6888-44ff-8e94-6744766bf026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:57.976419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:58.009507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 705/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Sep 13 07:51:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:51:58.017265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:58.019697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:58.129527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245de038-786f-4d19-bed5-e31d1e82fd73 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:51:58.172266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:58.210985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:51:58.211974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:58.212248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:58.212512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:58.214732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:58.339344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:51:58.340839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169c8f-c278-4fd6-ac7f-0fd650467adb service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:51:58.362158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 668/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:51:59.477382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:59.477919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:59.478035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:59.479394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 706/1374] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:51:59.484852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705c056-61ed-4cb4-8c03-f28a6a7e502f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:51:59.485273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705c056-61ed-4cb4-8c03-f28a6a7e502f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:51:59.485381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705c056-61ed-4cb4-8c03-f28a6a7e502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:51:59.551260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1705c056-61ed-4cb4-8c03-f28a6a7e502f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:51:59.553146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1705c056-61ed-4cb4-8c03-f28a6a7e502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:51:59.574244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:51:59.653077 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 669/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:51:59.943992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:51:59.945573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:00.103461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99545adb-767b-4cea-92b3-21cef5ccc80e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:00.156724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:00.220609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:00.221214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:00.221489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:00.221775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:00.223433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:00.316158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:00.318555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11948e0-c6e4-444d-9fed-636b9be6d252 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:00.337498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 707/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:02.529958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:02.530489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:02.530489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:02.532080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 670/1377] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:02.537726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d6eb7-275b-496e-af56-ea4cbe90ead7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:02.538165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d6eb7-275b-496e-af56-ea4cbe90ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:02.538350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d6eb7-275b-496e-af56-ea4cbe90ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:02.610025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8b6d6eb7-275b-496e-af56-ea4cbe90ead7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:02.611238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8b6d6eb7-275b-496e-af56-ea4cbe90ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:02.618828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:02.720567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 708/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:03.012112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:03.013808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:03.166519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71cd3b-844d-4f61-953e-b6139331c5c8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:03.170503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:03.287101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:03.287870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:03.288233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:03.288598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:03.290289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:03.382332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:03.384087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1ba14-998e-4acb-9c37-6159589fe9a2 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:03.403142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 671/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:05.608602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:05.609615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:05.610221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:05.612441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 709/1380] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:05.618563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad19dc-c0ee-4bca-b5de-3c9ca90c5963 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:05.619025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad19dc-c0ee-4bca-b5de-3c9ca90c5963 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:05.619339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad19dc-c0ee-4bca-b5de-3c9ca90c5963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:05.633936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:05.695133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b1ad19dc-c0ee-4bca-b5de-3c9ca90c5963 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:05.696881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b1ad19dc-c0ee-4bca-b5de-3c9ca90c5963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:05.841894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 672/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:06.132158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:06.134327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:06.299995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:06.311350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090a98f-7199-4e3b-bf91-423cd87960f1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:06.424945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:06.425464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:06.425768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:06.426040 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:06.427614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:06.517618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:06.519265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7681eab-bb85-4fc3-a4ac-aeaa840e778b service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:06.529287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:06.538686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 710/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:08.753196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:08.753895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:08.753895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:08.755290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 673/1383] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:08.760578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe68747-6728-4766-8a01-fc2a237f3f13 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:08.761028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe68747-6728-4766-8a01-fc2a237f3f13 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:08.761135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe68747-6728-4766-8a01-fc2a237f3f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:08.833385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8fe68747-6728-4766-8a01-fc2a237f3f13 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:08.834688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8fe68747-6728-4766-8a01-fc2a237f3f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:08.944275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 711/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:09.227985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:09.230132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:09.323090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:09.385740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2979c42-7c12-4b72-8b9b-122d28fb0384 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:09.504277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:09.504844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:09.505078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:09.505343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:09.506974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:09.558712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:09.604841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:09.606501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9038dc-634e-4e92-8762-d716d5b010a4 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:09.625989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 674/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:12.048952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:12.049483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:12.049483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:12.051367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 712/1386] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:12.056879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f26a-b436-4172-96c0-45e74ceb8590 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:12.057322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f26a-b436-4172-96c0-45e74ceb8590 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:12.057435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f26a-b436-4172-96c0-45e74ceb8590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:12.123625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1e60f26a-b436-4172-96c0-45e74ceb8590 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:12.124900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1e60f26a-b436-4172-96c0-45e74ceb8590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:12.222574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 675/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:12.512805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:12.514394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:12.591774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:12.663398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e2480-4d10-4ea1-ae09-e1a93527d461 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:12.776688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:12.777209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:12.777479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:12.777762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:12.779345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:12.823810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:12.876082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:12.877663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f44a-3868-4f23-863c-753e01fb264f service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:12.903029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 713/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:13.707927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2c5bf-ee69-450b-976d-130b3f87f8be None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:13.708495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2c5bf-ee69-450b-976d-130b3f87f8be None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:13.708495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2c5bf-ee69-450b-976d-130b3f87f8be None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:13.709851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 676/1389] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:13.715069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b47c8e-9af4-4110-b98d-86b32ffd6477 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:13.715489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b47c8e-9af4-4110-b98d-86b32ffd6477 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:13.715619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b47c8e-9af4-4110-b98d-86b32ffd6477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:13.782257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-83b47c8e-9af4-4110-b98d-86b32ffd6477 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:13.783452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-83b47c8e-9af4-4110-b98d-86b32ffd6477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:13.883153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 714/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:14.185962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:14.186339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:14.186744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:14.188151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 677/1391] 10.222.0.15 () {54 vars in 966 bytes} [Tue Sep 13 07:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:14.194195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c802-5146-4b1d-8a8f-49080b54f1f7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:14.194474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c802-5146-4b1d-8a8f-49080b54f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:14.194743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2c802-5146-4b1d-8a8f-49080b54f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:14.210359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:14.271821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f2e2c802-5146-4b1d-8a8f-49080b54f1f7 None None] MFA Rules not processed for user `4ec0de2bbadf4dbd9a5b66992b382c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:14.273015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f2e2c802-5146-4b1d-8a8f-49080b54f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:14.352044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 715/1392] 10.222.0.15 () {58 vars in 1063 bytes} [Tue Sep 13 07:52:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:52:14.361063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:14.362588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:14.393894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:14.466696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d284974-155f-47f0-9834-1e31daff3786 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:14.541422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:14.542083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:14.542366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:14.542639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:14.544226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:14.623190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:14.656252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:14.657701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a74f7c-8174-4efc-aa51-3ac8520fcb1f service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:14.676635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 678/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:15.094920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5491-65d4-49e8-9f61-c527ead1bce7 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.095443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5491-65d4-49e8-9f61-c527ead1bce7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.095443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5491-65d4-49e8-9f61-c527ead1bce7 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.096722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 716/1394] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:15.102756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.103612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.104099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.172623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:15.173213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:15.279279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 679/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:15.425309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567c6-0f10-416f-9daa-38d712624634 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.425650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567c6-0f10-416f-9daa-38d712624634 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.425960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4567c6-0f10-416f-9daa-38d712624634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.494681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4a4567c6-0f10-416f-9daa-38d712624634 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:15.495874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4a4567c6-0f10-416f-9daa-38d712624634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:15.548845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:15.572602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:15.573862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:15.581992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 717/1396] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:52:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:15.616761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:15.625769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228820b-a4cd-4567-9064-28878db0e3ed None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.626137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228820b-a4cd-4567-9064-28878db0e3ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.626413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228820b-a4cd-4567-9064-28878db0e3ed None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.627889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 718/1397] 10.222.0.15 () {54 vars in 943 bytes} [Tue Sep 13 07:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:52:15.634976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.635539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.635813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.693176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d68706-40a4-4a68-b3b4-c09f917677b0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:15.711793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] MFA Rules not processed for user `89e32c33e2184f5c93cc526f085c5375`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:15.712995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:15.808448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:15.809041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.809321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.809595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.811230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:15.825556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 719/1398] 10.222.0.15 () {58 vars in 1040 bytes} [Tue Sep 13 07:52:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:15.835100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:15.844763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d8e48-27f6-4e1a-b419-5c1f39b12b1a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:15.848173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:15.871795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:15.909174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:15.911326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fe9fe-bfc5-4bc3-89db-43ec7e2142ea service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:15.930556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 680/1399] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:52:15.954651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:15.955276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:15.955554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:15.955820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:15.957724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:16.047435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:16.049234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707450d-d70a-4d1d-8b6a-96695719de8e service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:16.059367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 720/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 07:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 07:52:18.187471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:18.187994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:18.187994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:18.189291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 681/1401] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:18.195209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f822eb-800a-47cd-8f22-de2678978d46 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:18.195692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f822eb-800a-47cd-8f22-de2678978d46 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:18.195774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f822eb-800a-47cd-8f22-de2678978d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:18.272258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-97f822eb-800a-47cd-8f22-de2678978d46 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:18.273839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-97f822eb-800a-47cd-8f22-de2678978d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:18.280950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:18.383305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 721/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:18.747326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:18.749074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:18.925296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683ced4-be52-4ed4-8f6f-4d4d78c9ffcc None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:18.943239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:19.070117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:19.071232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:19.071515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:19.071788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:19.073516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:19.173018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:19.174655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad10ec-e04b-4dba-a4bf-8cd9e14edbdd service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:19.194747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 682/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:21.413805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:21.413805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:21.413805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:21.419113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 722/1404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:21.420653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9f1c-0a41-4abf-9561-850b1d18a880 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:21.421101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9f1c-0a41-4abf-9561-850b1d18a880 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:21.421208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e9f1c-0a41-4abf-9561-850b1d18a880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:21.442346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:21.495025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1b3e9f1c-0a41-4abf-9561-850b1d18a880 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:21.496326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1b3e9f1c-0a41-4abf-9561-850b1d18a880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:21.619485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 683/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:21.916518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:21.918063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:22.070261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:22.085672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd34dcf-0bc5-4dd3-bd27-7cc2cade0323 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:22.209530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:22.210202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:22.210474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:22.210742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:22.212426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:22.313680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:22.315597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7315f-e04a-40e9-a283-6e0c8987026a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:22.326515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:22.337659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 723/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:24.539691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:24.540293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:24.540293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:24.541583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 684/1407] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:24.546990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0edf0-47a9-404e-9331-4bb85c2292d5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:24.547404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0edf0-47a9-404e-9331-4bb85c2292d5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:24.547590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0edf0-47a9-404e-9331-4bb85c2292d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:24.618415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-02c0edf0-47a9-404e-9331-4bb85c2292d5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:24.619649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-02c0edf0-47a9-404e-9331-4bb85c2292d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:24.723455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 724/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:52:25.012937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:25.015679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:25.132139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:25.195994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee772a8-6b65-4d03-8155-ec33209fd2ad None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:25.324522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:25.325135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:25.325414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:25.325744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:25.327481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:25.392953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:25.433168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:25.434959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a81ed-0145-4ee7-8550-cd501252d069 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:25.454398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 685/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:34.720956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=67 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:35.451641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=20 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:38.904994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa366-2f24-4e11-8afe-1ab8e6bfc8be None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:38.905483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa366-2f24-4e11-8afe-1ab8e6bfc8be None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:38.905651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffa366-2f24-4e11-8afe-1ab8e6bfc8be None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:38.906910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 725/1410] 10.222.0.15 () {54 vars in 918 bytes} [Tue Sep 13 07:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:38.911852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7aa6-ab6c-4e60-aa1f-e0d4ce03d6e4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:38.912291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7aa6-ab6c-4e60-aa1f-e0d4ce03d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:38.912424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7aa6-ab6c-4e60-aa1f-e0d4ce03d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:38.991974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8dee7aa6-ab6c-4e60-aa1f-e0d4ce03d6e4 None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:38.993191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8dee7aa6-ab6c-4e60-aa1f-e0d4ce03d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:39.098300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 686/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Sep 13 07:52:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:39.112295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ead8-fd63-4f02-ad65-0f0191940b2d None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:39.112579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ead8-fd63-4f02-ad65-0f0191940b2d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:39.112857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ead8-fd63-4f02-ad65-0f0191940b2d None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:39.114765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 726/1412] 10.222.0.15 () {54 vars in 918 bytes} [Tue Sep 13 07:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:39.119688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9997-be07-4c6b-9d28-8775f80b2fa7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:39.119971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9997-be07-4c6b-9d28-8775f80b2fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:39.120242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b9997-be07-4c6b-9d28-8775f80b2fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:39.193558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:39.196530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-407b9997-be07-4c6b-9d28-8775f80b2fa7 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:39.198356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-407b9997-be07-4c6b-9d28-8775f80b2fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:39.313127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 687/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Sep 13 07:52:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:41.890646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:41.891046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:41.891191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:41.892523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 727/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:41.898380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27ccd4-5c07-40e3-83d7-5465b1099ab7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:41.898823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27ccd4-5c07-40e3-83d7-5465b1099ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:41.898938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27ccd4-5c07-40e3-83d7-5465b1099ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:41.964487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7b27ccd4-5c07-40e3-83d7-5465b1099ab7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:41.965707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7b27ccd4-5c07-40e3-83d7-5465b1099ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:42.020311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:42.061836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 688/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:42.355418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:42.356936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:42.499931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7b302-27e3-4a13-82bc-ece5764eb3af None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:42.580477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:42.621581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:42.622147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:42.622423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:42.622687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:42.624321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:42.712738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:42.714359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d4d4a-03d8-4e53-8542-be02179d96fc service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:42.732203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 728/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:43.861346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:43.861938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:43.861938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:43.863793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 689/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:43.880941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614627a-29aa-4a75-9c58-162eddc56732 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:43.881454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614627a-29aa-4a75-9c58-162eddc56732 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:43.881562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614627a-29aa-4a75-9c58-162eddc56732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:43.957696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d614627a-29aa-4a75-9c58-162eddc56732 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:43.958908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d614627a-29aa-4a75-9c58-162eddc56732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:43.985193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:44.055178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 729/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:44.205832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:44.207230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:44.352095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af56c6-61ad-44e4-9d11-b38ea50830fe None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:44.392218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:44.464193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:44.464726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:44.464999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:44.465266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:44.466856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:44.552607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:44.554197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e8190-ec1c-483d-a7b0-2f468b201512 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:44.572332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 690/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:45.707457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:45.708529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:45.708529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:45.709889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 730/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:52:45.716040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099104-952b-4349-85fb-3bf65469fe64 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:45.716449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099104-952b-4349-85fb-3bf65469fe64 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:45.716585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099104-952b-4349-85fb-3bf65469fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:45.769432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:45.792230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4a099104-952b-4349-85fb-3bf65469fe64 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:45.793432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4a099104-952b-4349-85fb-3bf65469fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:45.889285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 691/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:46.046390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:46.047896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:46.191508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671ff8-081f-4a14-979a-ecfc6d4a9e7b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:46.203925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:46.304006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:46.304541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:46.304814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:46.305082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:46.306964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:46.393353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:46.394955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561060ff-b525-4f76-b79c-bfe9d3b4010e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:46.412207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 731/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:47.513608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:47.514108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:47.514205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:47.515534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 692/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:47.522298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39360f-d966-45f1-87d8-87c515bf28d2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:47.522734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39360f-d966-45f1-87d8-87c515bf28d2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:47.522830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39360f-d966-45f1-87d8-87c515bf28d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:47.542300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:47.597967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1b39360f-d966-45f1-87d8-87c515bf28d2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:47.599236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b39360f-d966-45f1-87d8-87c515bf28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:47.693704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 732/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:47.841057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:47.842685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:47.953623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:47.988479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f7b84-9afb-4130-a601-8b405c442ef5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:48.101104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:48.101635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:48.101942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:48.102214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:48.103800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:48.179749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:48.193852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:48.195455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403effb-7d17-408c-9fc0-ec70da6a2a05 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:48.213043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 693/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:49.328081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:49.328619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:49.328733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:49.330071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 733/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:49.336388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2714-961a-45bb-b71c-faae958708b6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:49.336808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2714-961a-45bb-b71c-faae958708b6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:49.336964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf2714-961a-45bb-b71c-faae958708b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:49.412742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1aaf2714-961a-45bb-b71c-faae958708b6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:49.414095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1aaf2714-961a-45bb-b71c-faae958708b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:49.510363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 694/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:49.658505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:49.659935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:49.743154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:49.804686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078d90c-2bbb-4c92-8615-eb21d2e0faa0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:49.917047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:49.917602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:49.917908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:49.918179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:49.919756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:49.967238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:50.009258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:50.010885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cd5ad-7213-4c1b-b361-776bf4a71aa2 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:50.029049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 734/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:51.073540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:51.074120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:51.074120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:51.075471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 695/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:51.081861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f70d6-79d8-4955-b8af-7ca2aaa02255 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:51.082284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f70d6-79d8-4955-b8af-7ca2aaa02255 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:51.082405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f70d6-79d8-4955-b8af-7ca2aaa02255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:51.156650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ed7f70d6-79d8-4955-b8af-7ca2aaa02255 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:51.157803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ed7f70d6-79d8-4955-b8af-7ca2aaa02255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:51.253850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 735/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:51.403127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:51.404699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:51.453958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:51.550966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1842a-68aa-4562-b891-691c38efa856 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:51.664962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:51.665573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:51.665899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:51.666172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:51.667755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:51.674681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:51.763215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:51.764830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad8d35-0780-4c61-8969-58d2114a7c9a service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:51.782924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 696/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:52.835475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:52.836088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:52.836088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:52.837365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 736/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:52.843762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743cba-e04d-4eb2-aa7a-caf9b5c8f47e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:52.844217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743cba-e04d-4eb2-aa7a-caf9b5c8f47e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:52.844316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1743cba-e04d-4eb2-aa7a-caf9b5c8f47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:52.921422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b1743cba-e04d-4eb2-aa7a-caf9b5c8f47e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:52.922659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b1743cba-e04d-4eb2-aa7a-caf9b5c8f47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:52.990767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:53.022525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 697/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:53.172886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:53.174381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:53.191861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:53.322287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d2db-edf8-4d69-915f-217d51377f99 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:53.411982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:53.438211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:53.438756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:53.439033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:53.439302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:53.440891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:53.528904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:53.530541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f2d65-a8b1-4af7-9dba-2e7a23dde40e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:53.548111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 737/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:54.594551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:54.595100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:54.595100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:54.596485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 698/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:54.603150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a106be-60de-4854-88d6-624557b43d1a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:54.603585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a106be-60de-4854-88d6-624557b43d1a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:54.603681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a106be-60de-4854-88d6-624557b43d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:54.677634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d6a106be-60de-4854-88d6-624557b43d1a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:54.678848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d6a106be-60de-4854-88d6-624557b43d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:54.719379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:54.774717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 738/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:54.924638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:54.926175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:55.071103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41fced-46aa-4ed1-936f-908f36ea2369 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:55.124140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:55.185229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:55.185870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:55.186180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:55.186454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:55.188040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:55.278514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:55.280104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a55716-87a2-4b37-8232-796aa5ac3817 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:55.297373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 699/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:56.350836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:56.351433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:56.351433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:56.352777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 739/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:56.359057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529de6-1a32-4dde-a608-21c788dec655 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:56.359503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529de6-1a32-4dde-a608-21c788dec655 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:56.359611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529de6-1a32-4dde-a608-21c788dec655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:56.434893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-17529de6-1a32-4dde-a608-21c788dec655 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:56.436084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-17529de6-1a32-4dde-a608-21c788dec655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:56.438087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:56.532600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 700/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:56.683016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:56.684458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:56.829918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3109736-a0c6-449e-9097-b6a3877f5c7c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:56.856610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:56.943792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:56.944305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:56.944576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:56.944841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:56.946437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:57.036649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:57.038272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b34b1b-6988-426a-9019-e210bc178ec4 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:57.056466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 740/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:52:59.136228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:59.136782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:59.136782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:59.138239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 701/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:52:59.143634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b3de3-85d6-4d8d-8ff9-05f29ab909c9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:59.144080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b3de3-85d6-4d8d-8ff9-05f29ab909c9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:59.144175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b3de3-85d6-4d8d-8ff9-05f29ab909c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:59.185649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:59.210992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bc6b3de3-85d6-4d8d-8ff9-05f29ab909c9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:52:59.212180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bc6b3de3-85d6-4d8d-8ff9-05f29ab909c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:59.312780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 741/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:52:59.608557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:52:59.610190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:59.714143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc0165-519f-49ac-949e-ed9fb3bde643 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:52:59.746216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:52:59.787179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:52:59.787781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:52:59.788055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:52:59.788320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:52:59.789921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:52:59.897659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:52:59.899160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a827-fdd4-460c-aebd-d3fb47522e84 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:52:59.916863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 702/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:02.275377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:02.275921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:02.275921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:02.277240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 742/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:02.282572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859da2f-2b6a-448e-b808-9ff6c8ab54bb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:02.283018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859da2f-2b6a-448e-b808-9ff6c8ab54bb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:02.283126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859da2f-2b6a-448e-b808-9ff6c8ab54bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:02.348820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d859da2f-2b6a-448e-b808-9ff6c8ab54bb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:02.350037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d859da2f-2b6a-448e-b808-9ff6c8ab54bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:02.362520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:02.446560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 703/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:02.728803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:02.730446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:02.834344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c12e4-c66e-4864-ad00-9502e81a75de None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:02.846999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:02.909489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:02.910132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:02.910407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:02.910671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:02.912854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:03.018749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:03.020141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d2e95-78f5-4911-b231-5e0dfda3c11d service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:03.045057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 743/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:05.473525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:05.474097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:05.474097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:05.475986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 704/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:05.481294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25d06-d274-47ad-a849-9d6d77ba1f3f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:05.481761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25d06-d274-47ad-a849-9d6d77ba1f3f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:05.481862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed25d06-d274-47ad-a849-9d6d77ba1f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:05.529447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:05.547510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6ed25d06-d274-47ad-a849-9d6d77ba1f3f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:05.548710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6ed25d06-d274-47ad-a849-9d6d77ba1f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:05.643553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 744/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:05.925606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:05.927227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:06.030715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b4b33-a439-4d89-b74e-8ef5133289cd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:06.070812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:06.105132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:06.105752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:06.106044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:06.106313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:06.107898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:06.215397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:06.216747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2fc2-04cd-4d7a-b989-d93b4c7794f2 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:06.234329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 705/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:08.549613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:08.550215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:08.550215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:08.551549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 745/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:08.557685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369d8f-2af2-48fe-ba1e-84c8ad37c7ab None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:08.558372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369d8f-2af2-48fe-ba1e-84c8ad37c7ab None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:08.558523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369d8f-2af2-48fe-ba1e-84c8ad37c7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:08.625401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7e369d8f-2af2-48fe-ba1e-84c8ad37c7ab None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:08.626673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7e369d8f-2af2-48fe-ba1e-84c8ad37c7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:08.647075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:08.844030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 706/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:09.149822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:09.151409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:09.253037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189d30a-b238-4457-b247-c3e6ad20f5e4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:09.277869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:09.336083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:09.336701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:09.336979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:09.337317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:09.339151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:09.464385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:09.465775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e5a01-9af3-4e9a-a5bd-0fb1229c0e5a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:09.483640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 746/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:11.769836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:11.770351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:11.770351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:11.772211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 707/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:11.777537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faec6-bea6-4f87-8bab-a3eb1fc80f26 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:11.778024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faec6-bea6-4f87-8bab-a3eb1fc80f26 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:11.778121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faec6-bea6-4f87-8bab-a3eb1fc80f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:11.841117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:11.843952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7a8faec6-bea6-4f87-8bab-a3eb1fc80f26 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:11.845683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7a8faec6-bea6-4f87-8bab-a3eb1fc80f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:11.941513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 747/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:12.226725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:12.228274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:12.330808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063becda-dcbb-48e6-825c-3111ebbb519b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:12.379736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:12.405601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:12.406260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:12.406542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:12.406812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:12.408399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:12.516566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:12.517945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ce91a-8a46-4527-ba69-f2f3cab460c0 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:12.535398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 708/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:14.741830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:14.742435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:14.742435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:14.743701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 748/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:14.748883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34e9cf-8510-4e90-b518-31941361acb1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:14.749320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34e9cf-8510-4e90-b518-31941361acb1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:14.749434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34e9cf-8510-4e90-b518-31941361acb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:14.813588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5f34e9cf-8510-4e90-b518-31941361acb1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:14.814806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5f34e9cf-8510-4e90-b518-31941361acb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:14.844230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:14.914440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 709/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:15.212116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:15.213680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:15.314134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e3487-5743-467d-81f0-2a3dec789f77 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:15.342992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:15.386818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:15.387421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:15.387700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:15.387968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:15.389533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:15.498452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:15.499903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa2f57-1548-4560-b6cb-10b08f36df1a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:15.518269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 749/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:17.702609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:17.703203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:17.703203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:17.704472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 710/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:17.709855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a85a6-f88b-4390-89f5-660f8a383971 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:17.710305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a85a6-f88b-4390-89f5-660f8a383971 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:17.710409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a85a6-f88b-4390-89f5-660f8a383971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:17.776034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9e7a85a6-f88b-4390-89f5-660f8a383971 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:17.777162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9e7a85a6-f88b-4390-89f5-660f8a383971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:17.785326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:17.873046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 750/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:18.153006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:18.154545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:18.253826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abba98-d589-4c35-a42c-1cba3158c168 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:18.307989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:18.326190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:18.326789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:18.327063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:18.327328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:18.328887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:18.436380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:18.437718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fcd4e-8ace-4890-b892-6f213cda5602 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:18.455643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 711/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:20.624878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:20.625443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:20.625443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:20.626784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 751/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:20.632395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bbc7-de87-4027-9b43-0384fccd35d5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:20.632822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bbc7-de87-4027-9b43-0384fccd35d5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:20.632943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bbc7-de87-4027-9b43-0384fccd35d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:20.698641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a2a5bbc7-de87-4027-9b43-0384fccd35d5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:20.699842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a2a5bbc7-de87-4027-9b43-0384fccd35d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:20.737327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:20.797270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 712/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:21.078204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:21.079763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:21.183068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe981b0-a4d1-4836-af89-79edd74c55b0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:21.220639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:21.255139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:21.255754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:21.256036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:21.256811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:21.258563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:21.366137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:21.367564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf805-a87d-41fd-b83a-20630a664363 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:21.385843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 752/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:23.533873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:23.534401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:23.534401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:23.535740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 713/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:23.540996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84b07-8e1a-4cbf-a6bf-636de94dd2b1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:23.541393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84b07-8e1a-4cbf-a6bf-636de94dd2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:23.541540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84b07-8e1a-4cbf-a6bf-636de94dd2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:23.606572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cae84b07-8e1a-4cbf-a6bf-636de94dd2b1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:23.607784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cae84b07-8e1a-4cbf-a6bf-636de94dd2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:23.625199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:23.702540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 753/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:23.984939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:23.986498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:24.090076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff3a8c-7640-4093-84c1-7b42579230f8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:24.154719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:24.161858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:24.162984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:24.163260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:24.163526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:24.165119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:24.271196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:24.272635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77e9a5-09b5-44d6-ae72-63cf87094219 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:24.290473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 714/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:26.453669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b211-da37-4411-a805-207294232455 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:26.454270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b211-da37-4411-a805-207294232455 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:26.454270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b211-da37-4411-a805-207294232455 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:26.455587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 754/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:26.460806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f42df8-b5aa-40e4-93bb-5fabbc53c38c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:26.461248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f42df8-b5aa-40e4-93bb-5fabbc53c38c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:26.461357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f42df8-b5aa-40e4-93bb-5fabbc53c38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:26.525968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a3f42df8-b5aa-40e4-93bb-5fabbc53c38c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:26.527170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a3f42df8-b5aa-40e4-93bb-5fabbc53c38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:26.574103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:26.624129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 715/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:26.906997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b211-da37-4411-a805-207294232455 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:26.908574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce34b211-da37-4411-a805-207294232455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:27.010168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34b211-da37-4411-a805-207294232455 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:27.058550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:27.083779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:27.084895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:27.085195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:27.085464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:27.087104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:27.202137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:27.203481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34ad04-6324-404a-ba1b-cc78d14e5957 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:27.221551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 755/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:36.622650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=22 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:37.219448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=66 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:47.093144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b0ad-b69e-4af7-a3b2-06f12c001039 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:47.093792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b0ad-b69e-4af7-a3b2-06f12c001039 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:47.093792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b0ad-b69e-4af7-a3b2-06f12c001039 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:47.095049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 716/1471] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:47.100209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375ee-72a8-4263-8141-e439d48af882 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:47.100621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375ee-72a8-4263-8141-e439d48af882 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:47.100759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9375ee-72a8-4263-8141-e439d48af882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:47.281227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0d9375ee-72a8-4263-8141-e439d48af882 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:47.282504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0d9375ee-72a8-4263-8141-e439d48af882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:47.379860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 756/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:47.679845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:47.680124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:47.680396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:47.681715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 717/1473] 10.222.0.15 () {54 vars in 966 bytes} [Tue Sep 13 07:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:47.686851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b96f3-5fb2-4517-bc35-c5d2b21d0e71 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:47.687140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b96f3-5fb2-4517-bc35-c5d2b21d0e71 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:47.687412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b96f3-5fb2-4517-bc35-c5d2b21d0e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:47.761488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d59b96f3-5fb2-4517-bc35-c5d2b21d0e71 None None] MFA Rules not processed for user `4ec0de2bbadf4dbd9a5b66992b382c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:47.762713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d59b96f3-5fb2-4517-bc35-c5d2b21d0e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:47.783145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:47.839182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 757/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Tue Sep 13 07:53:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:47.846801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:47.848272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:47.951052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d3d17-2a55-4a30-aa00-9608706e9953 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:48.021719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:48.022339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:48.023099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:48.023363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:48.024925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:48.077848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:48.133966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:48.135410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03381d66-ebbd-4101-b8a5-6ee411e783dd service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:48.154090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 718/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:50.354230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:50.354786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:50.354786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:50.356092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 758/1476] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:50.361255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be122a2a-6e13-4450-8c8c-df8866603e20 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:50.361701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be122a2a-6e13-4450-8c8c-df8866603e20 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:50.361844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be122a2a-6e13-4450-8c8c-df8866603e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:50.427012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-be122a2a-6e13-4450-8c8c-df8866603e20 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:50.428139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-be122a2a-6e13-4450-8c8c-df8866603e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:50.523519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 719/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:50.791931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:50.793452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:50.916339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1df9f-bc67-4b0c-805e-7754b6fd082e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:50.967744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:51.014724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:51.015327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:51.015597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:51.015860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:51.016492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:51.018249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:53:51.018249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:53:51.018249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:53:51.019390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f09b5-b84f-4186-ad25-fd91176f34f8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:51.037772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 759/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 07:53:50 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:53:53.193387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:53.194022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:53.194022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:53.195271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 720/1479] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:53.200488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2d445-4def-4fc4-b2a5-d10738f52b8c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:53.200918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2d445-4def-4fc4-b2a5-d10738f52b8c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:53.201043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2d445-4def-4fc4-b2a5-d10738f52b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:53.266574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-91a2d445-4def-4fc4-b2a5-d10738f52b8c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:53.267779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-91a2d445-4def-4fc4-b2a5-d10738f52b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:53.365039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 760/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:53.658356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:53.660046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:53.670481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:53.804808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eceaaa-bd66-4f7b-bd2a-92465448153e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:53.882565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:53.914989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:53.915591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:53.915862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:53.916124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:53.917688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:54.004864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:54.006473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87f86e-e07d-438c-ab50-fb0fe88a8240 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:54.024303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 721/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:56.346785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:56.347358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:56.347358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:56.348666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 761/1482] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:56.354591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdff1b-1796-456d-8f83-3194300bcfb7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:56.355024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdff1b-1796-456d-8f83-3194300bcfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:56.355167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdff1b-1796-456d-8f83-3194300bcfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:56.424924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-47cdff1b-1796-456d-8f83-3194300bcfb7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:56.426158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-47cdff1b-1796-456d-8f83-3194300bcfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:56.472005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:56.521449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 722/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:56.808342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:56.809908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:56.834166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:56.912052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14995718-10ec-46f3-bd34-6f7b30896b0b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:53:56.984802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:53:56.985403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:56.985678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:56.985972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:56.987546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:57.060921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:53:57.095775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:53:57.097263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c034b6-fa4e-492e-ace3-c1a8d1e3f971 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:53:57.114619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 762/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:53:59.392476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:59.393053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:59.393053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:59.394403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 723/1485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:53:59.400308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0bee-b055-4283-a62d-aec03bc42f53 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:53:59.400703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0bee-b055-4283-a62d-aec03bc42f53 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:53:59.400881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c0bee-b055-4283-a62d-aec03bc42f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:53:59.466687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0f6c0bee-b055-4283-a62d-aec03bc42f53 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:53:59.467874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0f6c0bee-b055-4283-a62d-aec03bc42f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:59.563267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 763/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:53:59.879505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:53:59.881041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:53:59.990845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576fc6-d243-4e53-83b4-10c0873c0eae None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:54:00.064262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:00.065783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:54:00.066454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:00.066740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:00.067184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:00.069155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:00.175601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:54:00.176934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41d271-ea8d-4bc9-9652-7d2674a493ed service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:54:00.195152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 724/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:54:02.426660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:02.427240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:02.427240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:02.428608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 764/1488] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:54:02.434025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70e813-870a-480e-a62d-1cffc6e884a4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:02.434455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70e813-870a-480e-a62d-1cffc6e884a4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:02.434554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70e813-870a-480e-a62d-1cffc6e884a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:02.499610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5c70e813-870a-480e-a62d-1cffc6e884a4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:54:02.500804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5c70e813-870a-480e-a62d-1cffc6e884a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:02.552232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:02.597202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 725/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:54:02.885670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:54:02.887279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:02.922670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:02.990024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9ee95-191d-4522-9e03-cab1bdd7e103 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:54:03.062404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:54:03.063006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:03.063280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:03.063544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:03.065120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:03.144559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:03.173319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:54:03.174780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0914c-543a-44e5-b9ed-6985ca1ed274 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:54:03.192977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 765/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:54:05.595481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:05.596040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:05.596040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:05.597376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 726/1491] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 07:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:54:05.603472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496c69-21cc-4aca-805f-f9f5162c4829 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:05.603902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496c69-21cc-4aca-805f-f9f5162c4829 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:05.604039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1496c69-21cc-4aca-805f-f9f5162c4829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:05.670739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d1496c69-21cc-4aca-805f-f9f5162c4829 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:54:05.671939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d1496c69-21cc-4aca-805f-f9f5162c4829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:05.766060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 766/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 07:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:54:06.058461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:54:06.059984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:06.164486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f474-ebea-4fe7-8eca-02f75be80282 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:54:06.239050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:54:06.239655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:54:06.239927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:54:06.240192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:54:06.241781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:54:06.250776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:06.348799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:54:06.350257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadff00-7cbe-4f17-a752-1ecfa946414c service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:54:06.368590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 727/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:54:15.764557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=82 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:54:16.365823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=50 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:25.169405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:25.170034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:25.170034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:25.246966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:55:25.248164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:25.343076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 767/1494] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:55:25.351014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:25.351529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:25.351648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:25.427530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:55:25.428637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:25.522734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 728/1495] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:55:25.527318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:25.528549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:25.539796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea941527-9c30-43a9-a90e-efb473be7691 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:25.602027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:25.636917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:25.637531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:25.637835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:25.638109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:25.638745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:55:25.639046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:25.640045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:25.640045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:55:25.640045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:55:25.640932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:25.652407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 768/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:55:25.660684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:25.670294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e1ad0-fe21-4ae6-ac2b-4d3aea4de7cc None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:25.730927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:25.763872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:25.764467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:25.764745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:25.765013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:25.765643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:25.767388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:25.767388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:55:25.767388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:55:25.768569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:25.824524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 729/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:55:25 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:55:25.828650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:25.830156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:25.863506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0b09f-e10e-4476-bc67-e126f9fa4213 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:25.955839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:25.956441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:25.956714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:25.956981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:25.957609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:25.958652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:25.958652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:55:25.958652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:55:25.959840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:26.002035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 769/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:55:25 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:55:26.006064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:26.007586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.069136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12ff39-d087-468b-8ab6-9448f8ef7cdf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:26.150425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:26.161313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:26.161950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.162225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.162491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.163156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:26.164162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:26.164162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:55:26.164162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:55:26.165055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:26.177179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 730/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:55:26.181204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:26.183244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.194838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a638a-884b-474d-a6ce-9cf28a7b0613 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:26.219903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:26.288777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:26.289285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.289567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.289865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.290500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:26.291504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:26.291504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:55:26.291504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:55:26.292450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f2f1b2-eb67-4fce-b6f9-3a9e2c18b5e9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:26.304450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 770/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:55:26.309033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:26.310565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.343895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ee62e-312d-4e91-a95f-af85cdb4eac9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:26.436669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:26.437323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.437603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.437904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] PATH_INFO: `/v3/projects/f92d0bfe35fa43d58bddba4748a59d8c/users/a10b1d23b69e4f6298eb60e0cffc86ad/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.460123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92d0bfe35fa43d58bddba4748a59d8c, user_id=a10b1d23b69e4f6298eb60e0cffc86ad, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:26.461148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:26.461148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:55:26.461148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:55:26.463133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58704395-788a-4582-9572-03a69ac80b71 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:26.487592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 731/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:55:26 2022] PUT /identity/v3/projects/f92d0bfe35fa43d58bddba4748a59d8c/users/a10b1d23b69e4f6298eb60e0cffc86ad/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:55:26.500136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388bcf-bf21-44d1-a9b6-2d7c5a955379 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.500448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388bcf-bf21-44d1-a9b6-2d7c5a955379 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.500720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388bcf-bf21-44d1-a9b6-2d7c5a955379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.555608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f2388bcf-bf21-44d1-a9b6-2d7c5a955379 None None] MFA Rules not processed for user `a10b1d23b69e4f6298eb60e0cffc86ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:55:26.556793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f2388bcf-bf21-44d1-a9b6-2d7c5a955379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.633094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 771/1502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:55:26.646607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.646890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.647159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.648522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 732/1503] 10.222.0.15 () {54 vars in 964 bytes} [Tue Sep 13 07:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:55:26.653325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ff1-39ad-4be0-a49c-15ab1ba8d9a1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.653632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ff1-39ad-4be0-a49c-15ab1ba8d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.653960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946ff1-39ad-4be0-a49c-15ab1ba8d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.685434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:26.730661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f1946ff1-39ad-4be0-a49c-15ab1ba8d9a1 None None] MFA Rules not processed for user `c8e54e867eee421b8f230c259d3cce68`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:55:26.731854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f1946ff1-39ad-4be0-a49c-15ab1ba8d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.807110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 772/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Sep 13 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:55:26.816827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:26.818356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:26.867362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:26.922583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa9f1b-2a50-4075-9936-7b49fef4c0ed None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:26.995905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:26.996508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:26.996783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:26.997050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:26.998664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:27.101938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:27.103284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:27.106762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:27.121471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 733/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:55:27.167142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954b035-0fc4-47a8-982d-e150ad1d7744 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:27.167456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954b035-0fc4-47a8-982d-e150ad1d7744 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:27.167734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954b035-0fc4-47a8-982d-e150ad1d7744 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:27.168972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 773/1506] 10.222.0.15 () {52 vars in 830 bytes} [Tue Sep 13 07:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:55:27.172583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:27.173820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:27.271062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f5282-b7aa-4c45-8299-a7408c6ade42 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:27.363316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:27.363875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:27.364154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:27.364426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] PATH_INFO: `/v3/users/a10b1d23b69e4f6298eb60e0cffc86ad` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:27.377065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10b1d23b69e4f6298eb60e0cffc86ad)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:27.378125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:27.378125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:55:27.378125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:55:27.379338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:27.389418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:27.445467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10b1d23b69e4f6298eb60e0cffc86ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:27.454467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10b1d23b69e4f6298eb60e0cffc86ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:27.454979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10b1d23b69e4f6298eb60e0cffc86ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:27.455402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10b1d23b69e4f6298eb60e0cffc86ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:27.470524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10b1d23b69e4f6298eb60e0cffc86ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:27.498679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 734/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:55:27 2022] DELETE /identity/v3/users/a10b1d23b69e4f6298eb60e0cffc86ad => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:55:27.503044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:27.503517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:27.503618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:27.559806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:55:27.561037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:27.635033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:27.648085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 774/1508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:55:27.659896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:27.661119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:27.759479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130087d-47fd-4600-83d4-46e452b9d393 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:27.813669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:27.834327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:27.835061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:27.835342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:27.835611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:27.837187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:27.945311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:27.946780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49597406-ca93-4fbe-a67f-cb2212d57909 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:27.964663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 735/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:55:28.285996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:55:28.287261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:55:28.299173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a843d8a-2b9b-4594-a2b0-e3fb36f0fd05 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:55:28.400060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:55:28.400689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:55:28.400966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:55:28.401758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] PATH_INFO: `/v3/projects/f92d0bfe35fa43d58bddba4748a59d8c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:55:28.411512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92d0bfe35fa43d58bddba4748a59d8c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:55:28.412531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:55:28.412531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:55:28.412531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:55:28.413724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:55:28.472277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92d0bfe35fa43d58bddba4748a59d8c', 'request_id': 'req-0f46e047-6dbd-4259-85e1-d78f2e655f67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:28.486848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92d0bfe35fa43d58bddba4748a59d8c', 'request_id': 'req-0f46e047-6dbd-4259-85e1-d78f2e655f67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:28.495619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92d0bfe35fa43d58bddba4748a59d8c', 'request_id': 'req-0f46e047-6dbd-4259-85e1-d78f2e655f67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:28.517919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] The token cache is being invalidate because project f92d0bfe35fa43d58bddba4748a59d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:55:28.518576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:28.518730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0f46e047-6dbd-4259-85e1-d78f2e655f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:55:28.519977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 775/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:55:28 2022] DELETE /identity/v3/projects/f92d0bfe35fa43d58bddba4748a59d8c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:55:37.962417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=62 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:55:38.515892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:56:55.014417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:55.015020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:55.015020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:55.016302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 736/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:56:55.021044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04e1-3614-4ba2-8673-9913347838c1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:55.021481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04e1-3614-4ba2-8673-9913347838c1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:55.021591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f04e1-3614-4ba2-8673-9913347838c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:55.100395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3f3f04e1-3614-4ba2-8673-9913347838c1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:56:55.101578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f3f04e1-3614-4ba2-8673-9913347838c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:56:55.206009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 776/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:56:55.357225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:56:55.359343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:56:55.478892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f014246-e608-477b-b2f9-c6054dbace53 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:56:55.484849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:56:55.485454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:55.485751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:55.486041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:55.493989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:56:55.495572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068e5e5-d31a-4b01-8da1-48874bbb3b46 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:56:55.505461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 737/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:56:56.612226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09272d66-03f0-4bc0-8175-a2a2e49087d9 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:56.612758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09272d66-03f0-4bc0-8175-a2a2e49087d9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:56.612758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09272d66-03f0-4bc0-8175-a2a2e49087d9 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:56.614063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 777/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:56:56.620312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247838a-8263-4e66-a816-39f4952df469 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:56.620743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247838a-8263-4e66-a816-39f4952df469 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:56.620848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247838a-8263-4e66-a816-39f4952df469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:56.640562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9247838a-8263-4e66-a816-39f4952df469 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:56:56.641772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9247838a-8263-4e66-a816-39f4952df469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:56:56.649558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 738/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:56:56.798393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09272d66-03f0-4bc0-8175-a2a2e49087d9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:56:56.812632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09272d66-03f0-4bc0-8175-a2a2e49087d9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:56:56.818102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:56:56.818966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:56.819243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:56.819515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:56.827984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:56:56.829559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde07504-4fcb-4ad2-9c9a-1cd8ed50e2ba service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:56:56.838768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 778/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:56:57.000023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586ded0-6cec-4e3f-a8a0-2453aafb3ec4 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:57.000340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586ded0-6cec-4e3f-a8a0-2453aafb3ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:57.000625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586ded0-6cec-4e3f-a8a0-2453aafb3ec4 None None] PATH_INFO: `/v3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:57.002040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 739/1517] 10.222.0.18 () {54 vars in 925 bytes} [Tue Sep 13 07:56:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:56:57.007955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f10d8-aa49-4ff2-a441-4e12110c4675 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:56:57.008269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f10d8-aa49-4ff2-a441-4e12110c4675 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:56:57.008548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f10d8-aa49-4ff2-a441-4e12110c4675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:56:57.066484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8e4f10d8-aa49-4ff2-a441-4e12110c4675 None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:56:57.067661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e4f10d8-aa49-4ff2-a441-4e12110c4675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:56:57.084684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:56:57.139351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 779/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 07:56:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:57:00.654740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc96db-173b-47c6-b800-ee1ddc189b1e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:00.655184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc96db-173b-47c6-b800-ee1ddc189b1e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:00.655312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc96db-173b-47c6-b800-ee1ddc189b1e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:00.656641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 740/1519] 10.222.0.15 () {54 vars in 961 bytes} [Tue Sep 13 07:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:00.661455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace43981-2494-4ec0-b55f-e0645f4ec8c5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:00.661923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace43981-2494-4ec0-b55f-e0645f4ec8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:00.662037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace43981-2494-4ec0-b55f-e0645f4ec8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:00.681464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ace43981-2494-4ec0-b55f-e0645f4ec8c5 None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:00.682786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ace43981-2494-4ec0-b55f-e0645f4ec8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:00.691136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 780/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Sep 13 07:57:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:00.698890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc96db-173b-47c6-b800-ee1ddc189b1e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:00.712344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc96db-173b-47c6-b800-ee1ddc189b1e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:00.717925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:00.718505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:00.718779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:00.719046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:00.727391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:00.728945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0f03-9fcd-406d-857b-d30fde382864 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:00.738535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 741/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:08.304150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7dbcb-6abb-42fa-88a3-a8ec2deb8bd1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:08.304689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7dbcb-6abb-42fa-88a3-a8ec2deb8bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:08.304689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7dbcb-6abb-42fa-88a3-a8ec2deb8bd1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:08.306041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 781/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:08.312520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb4ec-914c-4170-9945-46f6752d0723 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:08.312981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb4ec-914c-4170-9945-46f6752d0723 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:08.313112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb4ec-914c-4170-9945-46f6752d0723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:08.333214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0dbeb4ec-914c-4170-9945-46f6752d0723 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:08.334458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0dbeb4ec-914c-4170-9945-46f6752d0723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:08.342297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 742/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:08.504462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7dbcb-6abb-42fa-88a3-a8ec2deb8bd1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:08.519439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7dbcb-6abb-42fa-88a3-a8ec2deb8bd1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:08.524985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:08.525580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:08.525903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:08.526183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:08.534184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:08.535766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e232e-73b6-424d-9cba-1d43daffb15f service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:08.544976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 782/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:09.604602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c396-5108-4dd0-89bc-232224d9be9e None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:09.605178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c396-5108-4dd0-89bc-232224d9be9e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:09.605178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c396-5108-4dd0-89bc-232224d9be9e None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:09.606525 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 743/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 07:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:09.612587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b724a0-2208-4271-a454-982b0b1540f0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:09.613023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b724a0-2208-4271-a454-982b0b1540f0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:09.613147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b724a0-2208-4271-a454-982b0b1540f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:09.632496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-11b724a0-2208-4271-a454-982b0b1540f0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:09.633687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-11b724a0-2208-4271-a454-982b0b1540f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:09.642081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 783/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 07:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:09.795121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579c396-5108-4dd0-89bc-232224d9be9e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:09.809442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579c396-5108-4dd0-89bc-232224d9be9e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:09.815404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:09.815995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:09.816277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:09.816546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:09.824931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:09.827161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5634a-e55f-4089-a199-2ec80964a299 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:09.836530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 744/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:13.919465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:13.920011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:13.920011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:13.921336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 784/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:13.926705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae134b7-5200-4cb7-8493-3adbec3c48dc None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:13.927110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae134b7-5200-4cb7-8493-3adbec3c48dc None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:13.927249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae134b7-5200-4cb7-8493-3adbec3c48dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:13.946701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9ae134b7-5200-4cb7-8493-3adbec3c48dc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:13.947889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9ae134b7-5200-4cb7-8493-3adbec3c48dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:13.956128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 745/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:14.243707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:14.245400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:14.323618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff866a-1448-4f65-8961-106cacaf7659 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:14.327789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:14.328375 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:14.328652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:14.328920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:14.337705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:14.339154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d983f-d17a-4e6c-913d-68855fa988fe service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:14.348608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 785/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:16.491688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce5eb5-4b8a-494f-88c1-35957fe9c5dd None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:16.492317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce5eb5-4b8a-494f-88c1-35957fe9c5dd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:16.492317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce5eb5-4b8a-494f-88c1-35957fe9c5dd None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:16.493591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 746/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:16.499519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa913d83-ba72-476a-8375-726c581205c9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:16.499974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa913d83-ba72-476a-8375-726c581205c9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:16.500084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa913d83-ba72-476a-8375-726c581205c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:16.519831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fa913d83-ba72-476a-8375-726c581205c9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:16.521015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa913d83-ba72-476a-8375-726c581205c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:16.528723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 786/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:16.816129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce5eb5-4b8a-494f-88c1-35957fe9c5dd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:16.827711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce5eb5-4b8a-494f-88c1-35957fe9c5dd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:16.831170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:16.831762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:16.832038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:16.832312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:16.841189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:16.842653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b311e-3178-4f8f-b6e4-7f2068b14003 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:16.851890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 747/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:20.464526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:20.465064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:20.465064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:20.466929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 787/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:20.472063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ec8fb-a4aa-4db8-8b95-461469ca16f0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:20.472458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ec8fb-a4aa-4db8-8b95-461469ca16f0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:20.472607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ec8fb-a4aa-4db8-8b95-461469ca16f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:20.480115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=81 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:57:20.493391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f67ec8fb-a4aa-4db8-8b95-461469ca16f0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:20.494612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f67ec8fb-a4aa-4db8-8b95-461469ca16f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:20.502317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 748/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:20.806089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:20.807784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:20.893604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:57:20.907639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf0b9-5990-479b-b2d1-fb03f0f74eb3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:20.911355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:20.911961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:20.912234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:20.912502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:20.921924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:20.923355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23589c-1572-42b7-8e50-1e9f706f07e5 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:20.932881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 788/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:23.053414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:23.054005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:23.054005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:23.055328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 749/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 07:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:57:23.061176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26db15-6f92-422c-ab15-e814ffec1c9d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:23.061625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26db15-6f92-422c-ab15-e814ffec1c9d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:23.061722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26db15-6f92-422c-ab15-e814ffec1c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:23.081448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fe26db15-6f92-422c-ab15-e814ffec1c9d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:57:23.082666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fe26db15-6f92-422c-ab15-e814ffec1c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:23.090612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 789/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 07:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:57:23.394229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:57:23.395900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:57:23.409158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b1c61-e337-437a-bcee-2ad6e95d4d5b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:57:23.412621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:57:23.413209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:57:23.413481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:57:23.413771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:57:23.423133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:57:23.424562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c5fe0-9a6d-4f29-b5e0-ee58cadc8dd9 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:57:23.433935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 750/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:57:33.076585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=11 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:57:33.429800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=8 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:46.292749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ccc5-e069-45ae-ab6f-6187b7a24430 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.293402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ccc5-e069-45ae-ab6f-6187b7a24430 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.293402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ccc5-e069-45ae-ab6f-6187b7a24430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.319971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-93f0ccc5-e069-45ae-ab6f-6187b7a24430 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.321566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-93f0ccc5-e069-45ae-ab6f-6187b7a24430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.334650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 790/1540] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.344158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.344785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.344785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.355888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47585e-1211-4116-ba69-34d34ad637c0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.356219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47585e-1211-4116-ba69-34d34ad637c0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.357108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47585e-1211-4116-ba69-34d34ad637c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.374978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.376210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.391305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 751/1541] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.392049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3e47585e-1211-4116-ba69-34d34ad637c0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.393257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3e47585e-1211-4116-ba69-34d34ad637c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.405205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.405414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 791/1542] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.415549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.415850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.416098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.416786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4d197-b090-4615-b930-91e2bc64238b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.427032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.427757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.428041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.428314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.428981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:46.429288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.430344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:46.430344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:58:46.430344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:58:46.431241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a031d-ad7f-43f4-bc06-3c7d6aa75ef0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.449710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 752/1543] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:46.454119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.455282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.455435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.455736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.456017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.471600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 792/1544] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:58:46.472483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.493437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441e0c1-7056-4460-b3be-7fd096210e36 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.499046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.500581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.507902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.508535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.508911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.509189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.509952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.510981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:46.510981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:58:46.510981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:58:46.512168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5b3f5-844c-473f-9a79-7673ef4c6c25 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.522398 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 753/1545] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:58:46.523083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.536158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f069b-dedd-4d72-9604-1f0c90a05bd9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.549911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.550496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.550781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.551055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.551740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:46.552408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.554090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121bcf0-b03c-4367-885b-b12369db6a55 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.578542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 754/1546] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:46.580606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.581027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.581520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.597175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 793/1547] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:46.598544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e10ed-1ee3-4d19-ac6f-7eb9e375390a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.598825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e10ed-1ee3-4d19-ac6f-7eb9e375390a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.599093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e10ed-1ee3-4d19-ac6f-7eb9e375390a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.611584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.613551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.631800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-005e10ed-1ee3-4d19-ac6f-7eb9e375390a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.633069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-005e10ed-1ee3-4d19-ac6f-7eb9e375390a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.684999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 755/1548] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.684999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.691704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586618dd-3b1b-41dd-b9cb-68c6f36d705b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.696682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.697282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.697571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.697899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.698444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 794/1549] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.699870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.700181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.700452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.705246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.707274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa0ed5-87d6-468f-9a23-60fed6cc7b9e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.720066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.721284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.729462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 795/1550] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.730457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.739853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 756/1551] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:46.740789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333c281-2792-4b1a-bcab-728c9bbaf4f6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.741422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea9f57-2879-409d-bab9-f8634351fece None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.741644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea9f57-2879-409d-bab9-f8634351fece None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.741954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea9f57-2879-409d-bab9-f8634351fece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.763842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bbea9f57-2879-409d-bab9-f8634351fece None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.765599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bbea9f57-2879-409d-bab9-f8634351fece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.799758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.800416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.800639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.800905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.801528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.803471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d770f-7608-4a25-8943-d39034a62ad3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.825342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 796/1552] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:46.826838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.827197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.827472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.832442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 757/1553] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.833691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21bee-6195-4bea-916e-e694ec3be003 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.834005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21bee-6195-4bea-916e-e694ec3be003 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.834279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21bee-6195-4bea-916e-e694ec3be003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.849416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.850638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.854788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-91d21bee-6195-4bea-916e-e694ec3be003 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.855961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-91d21bee-6195-4bea-916e-e694ec3be003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.859665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 797/1554] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.860803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.864690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 758/1555] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.870669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9948b-d171-40e8-9a74-06d811e0426e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.871715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.872115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.872323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.875466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.876057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.876335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.876608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.877308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:46.877550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.879174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f1b46-7783-47b0-8d83-09d01229984c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.892223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 798/1556] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:46.892394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.893463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.893673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.893784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.897572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.907284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 759/1557] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.908331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.916786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.917914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa363fd-86a8-4005-8fdb-1c2386be4ab3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.918033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.922691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.923287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.923568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.923837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.924469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.925491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:46.925491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:58:46.925491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:58:46.926746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9ac58-da2c-4585-8381-7b2958a7ec42 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.929098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 799/1558] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.932797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:46.940840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25179f-09b7-454f-a1e6-d1d5385da526 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:46.947733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:46.948331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.948541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 760/1559] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:46.948908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.949183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.949861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:46.950547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1cb9-75ce-4e83-a1e9-6ccda61e0ace None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.950842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1cb9-75ce-4e83-a1e9-6ccda61e0ace None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.951119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f1cb9-75ce-4e83-a1e9-6ccda61e0ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.953994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dadb11-7c6e-49c4-8b6d-8eeb277d248b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:46.955469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:46.965999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 800/1560] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:46.967731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.968781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.969594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.971391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bc5f1cb9-75ce-4e83-a1e9-6ccda61e0ace None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.972605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bc5f1cb9-75ce-4e83-a1e9-6ccda61e0ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.980549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 761/1561] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:46.981951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:46.982246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:46.982526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:46.988945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:46.989966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:46.990132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:46.999447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 801/1562] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:47.000212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.004096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:47.005284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:47.011981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8df421-a682-4a92-ad65-c6024208bac0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.013817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 762/1563] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:47.017426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.017543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.018157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.018434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.018699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.019333 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.021247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.023921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bfb6b-ffe8-47fa-93e8-404dd8c01e2d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.029059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.029644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.029960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.030236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.030881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:47.031185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.032815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.044988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 763/1564] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:47.045833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.050381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 802/1565] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:46 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.051339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.055915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d265d36-4dd2-462e-ae9e-5c4789f5151b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.111620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.112228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.112512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.112786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.113402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395322e8-ee94-4f52-964c-e05d86a80e33 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.113517 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:47.113721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.115357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.118083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.118684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.118964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.119235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.119876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.121467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.127356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 764/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:47.128178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.138236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850577a-24d0-4e01-b3cd-ad659e21f59c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.140654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 803/1567] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.141714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.142876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.143478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.143830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.144110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.144749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.146456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.151400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7de86-5a29-48f6-834a-4ee852e6e513 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.157036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 765/1568] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.157864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.160122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.160725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.161037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.161311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.161991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:47.162447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.163892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.167069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70652c-ec18-44c4-a7d8-604296af2f9f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.171634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.172200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.172476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.172742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.173360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:47.173655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.175221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.182957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 804/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:47.182957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.187491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 766/1570] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:47.187639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938bb1-d264-4885-bce6-f49d529ef0e0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.188450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.192190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.192775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.193049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.193313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.193973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:47.194269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.196199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.199933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333f638-ff77-40d3-be8a-a3e95c7a69f7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.205190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.205964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.206369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.206657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.207318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.215600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 805/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:47.216341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.216446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.219546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f395-3fcb-4c30-99ad-9702a7e9325f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.224582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.225170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.225452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.225726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.226382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.228296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.240594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 767/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.240594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.255535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 806/1573] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.255535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.263722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9f0cc-6edd-458e-83d1-93e45d2da71e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.306895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bd741-21dc-4578-81e3-86dabcd9753b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.312202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.312881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.313225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.313500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.314172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.316082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.321056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.321670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.321998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.322269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.322915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.324503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.335222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 807/1574] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.336279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.344902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 768/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.345695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.346298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d56148-815d-411c-bbe8-e13636f8db5c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.358190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:47.363892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:47.405933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.406765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.407052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.407328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] PATH_INFO: `/v3/projects/27168502db1845468d219bbdcf68f42b/users/49bd70d7364849a68ae503d7ba6cc9f1/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.411186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a4be9-98c5-43ce-8b9d-d996c043befa admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.415842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.416450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.416729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.417004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.417646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.419667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.422477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27168502db1845468d219bbdcf68f42b, user_id=49bd70d7364849a68ae503d7ba6cc9f1, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.423520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:47.423520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:58:47.423520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:58:47.425457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.450007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 808/1576] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:47 2022] PUT /identity/v3/projects/27168502db1845468d219bbdcf68f42b/users/49bd70d7364849a68ae503d7ba6cc9f1/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:47.450007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.461108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 769/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.461108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.507369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9416d-4d3f-45b0-b9d1-315cf97bd566 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.520208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08984e2-81d5-4b79-b9be-068cb13802cb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.524934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.525536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.525853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.526128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.526766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.528720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.555099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 770/1578] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.555897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.563447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.563999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.564279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.564551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.565183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.567149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.594202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 809/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/projects => generated 405 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.595017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.620168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dd213-2a8a-48a6-9002-47861ab1678c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.628368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:47.650260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:47.661135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2b47-c3f1-4b43-aba7-71183300ac87 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.666177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.666831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.667111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.667380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.668071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.670263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.678362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.678976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.679254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.679640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.679972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 810/1580] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.680279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.680731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.682285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.690453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41c3d-1754-4421-af2f-fea3ea406927 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.695060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.695945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.696204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.696533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] PATH_INFO: `/v3/projects/074dbd6970034af1b02996d318d42e88/users/0d7a4ca3d0904c1f8e3b9622fea8af8c/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.706939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 771/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 282 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.710676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.712184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074dbd6970034af1b02996d318d42e88, user_id=0d7a4ca3d0904c1f8e3b9622fea8af8c, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.714896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8038a8-16c4-47a9-a1a8-3d555e857876 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.720162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034f2b5-b196-4e99-b2ce-1166fee48d2f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.725254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.726036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.726323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.726614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.727224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.728224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 811/1582] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:47 2022] PUT /identity/v3/projects/074dbd6970034af1b02996d318d42e88/users/0d7a4ca3d0904c1f8e3b9622fea8af8c/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:47.729134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.729578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.729902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.730177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.756391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 772/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.757207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.760089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] MFA Rules not processed for user `49bd70d7364849a68ae503d7ba6cc9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:47.761328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:47.766740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0683-7b77-4427-9b1d-4ad502b1bbd4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.823240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.823924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.824204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.824471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.825112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.826797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 812/1584] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:47.827264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.827621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.844556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b7d32-5694-4bdf-9380-db3c43835652 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.849111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 773/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.852385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.852385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.852611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.852721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.852721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.852721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.852976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.860789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda5a1-f227-4155-ac34-3b39d805d6f5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.865335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.866026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.866370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.866735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.867500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.870378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.878432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 813/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.878432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.890525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f45b3c2-0240-4904-a857-f1da98084976 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.892387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 774/1587] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:47.893225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.896132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.896945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.897382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.897899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.898958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.901231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3767-b951-4854-8ff9-8f89fb70e2e9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.903583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19513530-0d55-40dc-8534-762de6a9cf5d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.908225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.908816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.909092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.909368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.910033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.911730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.912253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 814/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.915512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.915512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.915512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.923037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 775/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.923838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.932819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:47.933715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3943-3cd3-4389-86a2-d549985629dc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.934165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:47.938944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.939670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.939951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.940223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.940886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.942415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 815/1590] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:47.942545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.943178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.956743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 776/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.957556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:47.972733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fdce0-aecf-4c48-a4f1-ccfd18d55318 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:47.977459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:47.978084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:47.978388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:47.978657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:47.979316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:47.980940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:47.990967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 777/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:47.991743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.001980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48880c-1591-41e2-89e2-512be193081b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.006550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.007126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.007407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.007682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.008304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.010035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.010659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2d61-ee04-40ef-a998-191ae8763727 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.015060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.016037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.016037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.016271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.021397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 778/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.022710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.024601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.026143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.031327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:48.033873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5f088-64f8-47dc-b959-4cf1807f9219 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.036373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 816/1594] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:48.037374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.039127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.039757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.039983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.040255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.041035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.042696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.048747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b7d15-86cf-4258-95bf-143d17954c65 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.054141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 779/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.055071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.058088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.059006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.060314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.060671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] PATH_INFO: `/v3/projects/3414a292093541209ace108ad8bb6be4/users/272f5244f9724e00bd35dd09fedc6b67/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.067698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b195b-8e7d-42ee-83ea-3001e9fe7f3a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.073627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.074328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.074709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.075649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.076392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.078143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.078642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3414a292093541209ace108ad8bb6be4, user_id=272f5244f9724e00bd35dd09fedc6b67, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.081264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.089549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 780/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.090537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.099072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 817/1597] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/3414a292093541209ace108ad8bb6be4/users/272f5244f9724e00bd35dd09fedc6b67/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.100671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.103518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:48.151568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b2dd-a557-4ae2-bd25-04843605f241 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.174803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec02d58-318e-46dc-818f-96199dee16ab admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.180302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.181027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.181402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.181799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.182679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.185187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.196272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 818/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.197755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.208321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4d049-e12a-4354-af6b-326f7ccb42d2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.209248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.209905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.210191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.210462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.213668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.214599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.214880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.215148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.215942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.218297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.219855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.221323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.229781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 819/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.230778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.234387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 781/1600] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:48.235272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.243768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24e5ed-f245-4704-8a51-c39dbb74d488 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.244555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a619d-4b68-4f96-8402-058503cc133e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.250111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.250338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.250721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.250941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.251053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.251220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.251354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.251491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.251894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.252134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.253503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd72ea-40f1-4969-8c1e-66f286e75d6d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.253724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.269377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 782/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.269377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.269761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 820/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:48.269761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.269761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.269761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.287317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f108-bbf5-48bb-85c2-fbe8b03a95bf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.290725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:48.292993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:48.297221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.298939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.299545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.300430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] PATH_INFO: `/v3/projects/97efdf3e4d9d4aab9760afae95a6ddcf/users/53108e570b2c4d1bbd028c59b64b496b/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.305915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 821/1603] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:48.306262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.316858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ea35-865e-48e0-85be-dac9c2aa8a82 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.323304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.324438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.325626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.326029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] PATH_INFO: `/v3/projects/9f884173ca424e9480348995d9288164/users/afa6f32a8e9041429acb5ae15e01b6f1/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.330268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97efdf3e4d9d4aab9760afae95a6ddcf, user_id=53108e570b2c4d1bbd028c59b64b496b, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.336088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.345912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f884173ca424e9480348995d9288164, user_id=afa6f32a8e9041429acb5ae15e01b6f1, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.349191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.361856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 783/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/97efdf3e4d9d4aab9760afae95a6ddcf/users/53108e570b2c4d1bbd028c59b64b496b/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.363004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.378246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 822/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/9f884173ca424e9480348995d9288164/users/afa6f32a8e9041429acb5ae15e01b6f1/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.379078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.451618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1519d-a3dd-49f6-ab49-728943306597 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.488095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6dd-a596-461b-9123-78bb6e870db1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.494454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.495309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.495598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.495982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] PATH_INFO: `/v3/projects/85b34d4bdbe24b7bb1cb9c4d8923da04/users/c491a16b8c104c5193868fd6d182ca1f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.522271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b34d4bdbe24b7bb1cb9c4d8923da04, user_id=c491a16b8c104c5193868fd6d182ca1f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.525311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.542410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:48.545859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 823/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/85b34d4bdbe24b7bb1cb9c4d8923da04/users/c491a16b8c104c5193868fd6d182ca1f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.554058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.554058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:48.578606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.582330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.582330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.582330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] PATH_INFO: `/v3/projects/396240fee2a545858feb20fb12b58b19/users/0e8fc95d63834bd0a60930c36d128129/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.601716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396240fee2a545858feb20fb12b58b19, user_id=0e8fc95d63834bd0a60930c36d128129, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.608831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.643261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 784/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/396240fee2a545858feb20fb12b58b19/users/0e8fc95d63834bd0a60930c36d128129/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.644381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:48.657230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:48.683515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a501f8-5fbc-484f-8612-d4e0cada572e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.748539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42b70-188a-4f58-bb1b-d9347671e4c7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:48.754856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.755692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.756151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.756452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.806347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:48.808585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.809231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.809989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] PATH_INFO: `/v3/projects/673e7e0a20014808a399969e1d5e0d41/users/bada6717873d408a95058051f2589a89/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.846558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.849781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b16b5-2104-48b3-b5dd-61d0e0c88c07 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.857413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673e7e0a20014808a399969e1d5e0d41, user_id=bada6717873d408a95058051f2589a89, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:48.860556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12024a70-5ebb-4704-8504-44211e086df1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:48.865609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 785/1608] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:48.867534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057990b9-b1e0-4ab3-af30-259ca068e65f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.868049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057990b9-b1e0-4ab3-af30-259ca068e65f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.868562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057990b9-b1e0-4ab3-af30-259ca068e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.878477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 824/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:48 2022] PUT /identity/v3/projects/673e7e0a20014808a399969e1d5e0d41/users/bada6717873d408a95058051f2589a89/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:48.880445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2456d4-7391-47ba-af58-6570dba0ddf2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:48.880981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2456d4-7391-47ba-af58-6570dba0ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:48.881366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2456d4-7391-47ba-af58-6570dba0ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:48.911847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-057990b9-b1e0-4ab3-af30-259ca068e65f None None] MFA Rules not processed for user `53108e570b2c4d1bbd028c59b64b496b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:48.913609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-057990b9-b1e0-4ab3-af30-259ca068e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:48.922612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9d2456d4-7391-47ba-af58-6570dba0ddf2 None None] MFA Rules not processed for user `afa6f32a8e9041429acb5ae15e01b6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:48.923936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9d2456d4-7391-47ba-af58-6570dba0ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:48.995481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 786/1610] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.004147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13328b4-1a01-4c9c-ae9d-539fcce88432 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.004441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13328b4-1a01-4c9c-ae9d-539fcce88432 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.004773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13328b4-1a01-4c9c-ae9d-539fcce88432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.019122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:49.036878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 825/1611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.043188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.043489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.043763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.051821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f13328b4-1a01-4c9c-ae9d-539fcce88432 None None] MFA Rules not processed for user `c491a16b8c104c5193868fd6d182ca1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:49.054098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f13328b4-1a01-4c9c-ae9d-539fcce88432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:49.102502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] MFA Rules not processed for user `0e8fc95d63834bd0a60930c36d128129`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:49.103633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:49.115321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:49.167028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 787/1612] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.168277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0128f-0a9a-48ae-a929-2bc27925b89a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.168813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0128f-0a9a-48ae-a929-2bc27925b89a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.169325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0128f-0a9a-48ae-a929-2bc27925b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.193999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bbf0128f-0a9a-48ae-a929-2bc27925b89a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:49.195390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bbf0128f-0a9a-48ae-a929-2bc27925b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:49.219552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 826/1613] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.220776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.302386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 788/1614] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.303882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46c470-12d9-4fa5-ae73-233e3be1c22c None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.304267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46c470-12d9-4fa5-ae73-233e3be1c22c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.304641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46c470-12d9-4fa5-ae73-233e3be1c22c None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.310899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 789/1615] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 07:58:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:58:49.311863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46c470-12d9-4fa5-ae73-233e3be1c22c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.327455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46c470-12d9-4fa5-ae73-233e3be1c22c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.333378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.334192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.334591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.334951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.335690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.337976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0d099-93b0-42e1-9345-7018777d3ffb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.348936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e8f0c-248e-41b8-805c-27477613f65b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.362626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.363578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.365076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.365447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.366332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.370982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.393219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 790/1616] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/projects => generated 405 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:49.393219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200269-5e76-491b-b74d-d49addc5b7c7 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.393219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200269-5e76-491b-b74d-d49addc5b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.393219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200269-5e76-491b-b74d-d49addc5b7c7 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.394938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 791/1617] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 07:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:58:49.396444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.396599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.396950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.441799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 827/1618] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/projects => generated 365 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:49.446118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.480188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:49.482039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:49.575080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:49.583162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b81b-4404-4091-919a-621a9c507f71 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.588020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.592143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.592344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.592628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.611411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 792/1619] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.613653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.707081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:49.733668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.737322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70bdaa-8b98-4d28-b4cc-50ccb3393a22 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.751862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2ca7b-1ecb-4e18-8153-b3cbfac22ab9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.759546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 828/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:49.761882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.762592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.763734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.764593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.765288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.765812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.766999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.767718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.771581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.806708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:49.808278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:49.821056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 793/1621] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:49.822165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.824633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 829/1622] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:49.837210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.841422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2ac39-05b3-4c31-96ad-feb89f4ab16b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.846420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.847320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.847646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.847924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.848692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.850684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c3630-6404-454a-8e7b-5e831fe97b44 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.850834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.858218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.859278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.859740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.860264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.910221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 794/1623] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:49 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:49.912007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.937237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aabb5d-52f9-425c-9782-b4902615e198 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.949292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:49.950628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:49.951049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:49.951330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:49.951999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.953640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.959852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:49.962353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:49.975095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 830/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:49.982150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:49.989413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b69-86dc-4068-a866-9840bb907a7a service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:49.999944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.000682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.003313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.003732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.008177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 795/1625] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:50.008177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.032395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277866-b039-48f6-97a9-f360c667b43e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.045304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.046903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.048123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.048723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.050285 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.056305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.089623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 796/1626] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:50.091236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.118047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3226e-17c2-4bde-85ae-a4341e6fa88d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.131693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.136973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.137281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.138898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] PATH_INFO: `/v3/projects/e29a219bc4e74f4590fe5e44261e1821/users/55f857154b5d4d6aba2b0cdb657303fd/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.152546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.158491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.167018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29a219bc4e74f4590fe5e44261e1821, user_id=55f857154b5d4d6aba2b0cdb657303fd, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.170104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.181335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 831/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:50.188555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.196472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 797/1628] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:50 2022] PUT /identity/v3/projects/e29a219bc4e74f4590fe5e44261e1821/users/55f857154b5d4d6aba2b0cdb657303fd/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:50.201152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.216768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a816-15a7-4171-b21a-57d9b55b9d4e service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.351282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1a01a-8497-4cc1-b4e8-33ea21ff7044 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.358856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.359543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.359828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.360104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] PATH_INFO: `/v3/projects/72058af83ed0463b924b0ada17ee20b2/users/988bb7e503854780a95ba6919e7b3853/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.364268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.366272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.366570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.366848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.368145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.369768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.398910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] RBAC: Authorizing `identity:create_grant(project_id=72058af83ed0463b924b0ada17ee20b2, user_id=988bb7e503854780a95ba6919e7b3853, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.399127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 798/1629] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:50.400141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.403733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.416832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290de6eb-0f76-4e76-938a-f95f6a926314 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.422653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:50.433893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 832/1630] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:50 2022] PUT /identity/v3/projects/72058af83ed0463b924b0ada17ee20b2/users/988bb7e503854780a95ba6919e7b3853/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:50.436777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.437306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.438306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.438645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.439000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] PATH_INFO: `/v3/projects/e29a219bc4e74f4590fe5e44261e1821/users/55f857154b5d4d6aba2b0cdb657303fd/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.444720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29a219bc4e74f4590fe5e44261e1821, user_id=55f857154b5d4d6aba2b0cdb657303fd, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.452274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161919ed-e565-47d4-84ab-d6ffba11fb03 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.469430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:50.477756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 799/1631] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:50 2022] PUT /identity/v3/projects/e29a219bc4e74f4590fe5e44261e1821/users/55f857154b5d4d6aba2b0cdb657303fd/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:50.491728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c2560-2e6d-4dc2-8f82-1559f64793c2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.492018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c2560-2e6d-4dc2-8f82-1559f64793c2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.492356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c2560-2e6d-4dc2-8f82-1559f64793c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.587177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3863c-e3e3-46d3-8bf0-6e85c96526be admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.594988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c71c2560-2e6d-4dc2-8f82-1559f64793c2 None None] MFA Rules not processed for user `55f857154b5d4d6aba2b0cdb657303fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:50.597530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c71c2560-2e6d-4dc2-8f82-1559f64793c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:50.719423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.721394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.722438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.722860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.724815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.725642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 800/1632] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:50 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:58:50.727207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e588b7-1a02-4151-aa8c-3a7564b601e5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.727501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e588b7-1a02-4151-aa8c-3a7564b601e5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.727779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e588b7-1a02-4151-aa8c-3a7564b601e5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.730697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2a983-86f7-4705-a454-cb2e6e50d0f9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.734684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 801/1633] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 07:58:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:58:50.735749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e588b7-1a02-4151-aa8c-3a7564b601e5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.766383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 833/1634] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:50.766383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f491d9-1a8b-4b1e-a32e-0a1c645f6630 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.766383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f491d9-1a8b-4b1e-a32e-0a1c645f6630 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.766383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f491d9-1a8b-4b1e-a32e-0a1c645f6630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.804893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e588b7-1a02-4151-aa8c-3a7564b601e5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.808887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.809556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.809870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.810147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.820253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.823732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.834992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-76f491d9-1a8b-4b1e-a32e-0a1c645f6630 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:50.836236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76f491d9-1a8b-4b1e-a32e-0a1c645f6630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:50.857216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 802/1635] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:50 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:50.858546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:50.978821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df51c2-82d4-48b7-ae04-709e58dadf46 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:50.978821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:50.978821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.978821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.980756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] PATH_INFO: `/v3/projects/72058af83ed0463b924b0ada17ee20b2/users/988bb7e503854780a95ba6919e7b3853/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.980756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72058af83ed0463b924b0ada17ee20b2, user_id=988bb7e503854780a95ba6919e7b3853, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:50.980756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44edef-bfd0-4fb3-8d6d-42a653367ac1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:50.980756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 803/1636] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:50 2022] PUT /identity/v3/projects/72058af83ed0463b924b0ada17ee20b2/users/988bb7e503854780a95ba6919e7b3853/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:50.980756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5eed-05d9-4335-b830-2b7f6f46dfd5 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5eed-05d9-4335-b830-2b7f6f46dfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5eed-05d9-4335-b830-2b7f6f46dfd5 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 804/1637] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 07:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:50.981678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.071753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] MFA Rules not processed for user `988bb7e503854780a95ba6919e7b3853`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:51.073065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:51.147556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 834/1638] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:58:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:51.149213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.149704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.150181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.175700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:51.177450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:51.218495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:51.248316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:51.259150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 805/1639] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:51.260412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:51.277673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 835/1640] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:58:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:51.279095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:51.349431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dd9d5-1af0-487e-b6eb-a043640abfbe None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:51.360145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:51.360824 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.361101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.361412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.377270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e922ed-727f-49ae-a15e-866f25f984de None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:51.381845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:51.385017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:51.393897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:51.394561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.394842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.395510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] PATH_INFO: `/v3/users/c491a16b8c104c5193868fd6d182ca1f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.398411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c491a16b8c104c5193868fd6d182ca1f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:51.401321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: WARNING py.warnings [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:51.401321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: warnings.warn(msg) Sep 13 07:58:51.401321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: Sep 13 07:58:51.402612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:51.405300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 806/1641] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:51.406715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:51.417605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a973b1-7f61-4043-b7c9-7f2d18233f6b service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:51.422979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:51.423595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.423877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.424152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.435956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:51.438347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:51.453373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 807/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:51.521646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:51.544462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1743-8eda-4beb-91ad-640e09adbe88 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:51.570071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:51.571200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.571582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.571959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.578987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c491a16b8c104c5193868fd6d182ca1f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:51.594540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c491a16b8c104c5193868fd6d182ca1f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:51.595224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c491a16b8c104c5193868fd6d182ca1f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:51.596483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c491a16b8c104c5193868fd6d182ca1f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:51.597772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:51.599556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7c395-187a-42aa-8a4c-a8e9cd6aebd0 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:51.617110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 808/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:51.631769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c491a16b8c104c5193868fd6d182ca1f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:51.688002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 836/1644] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:58:51 2022] DELETE /identity/v3/users/c491a16b8c104c5193868fd6d182ca1f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:51.693269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:51.693642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:51.694055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:51.762742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:51.774235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:52.056543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 809/1645] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:52.080229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:52.081046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:52.327454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:52.329354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:52.374989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:52.389326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aecd201-28fc-47fa-967e-96e272950b73 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:52.419874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:52.420726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:52.421013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:52.421287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:52.443559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:52.446708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:52.468304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 837/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:58:52.535678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba691f5-386c-44d6-b69c-4e7d28e8944b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:52.535678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:52.535678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:52.535678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:52.539448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:52.539448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:52.585323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:52.652577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:52.655026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be982283-6058-4b53-9c47-9d79af3e4f88 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:52.670662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 810/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:52.953389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:52.956182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:52.983901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a21692-7124-41d6-9744-7c30643d210a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:52.999281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:53.002928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:53.004152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:53.005685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] PATH_INFO: `/v3/users/0e8fc95d63834bd0a60930c36d128129` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:53.008716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8fc95d63834bd0a60930c36d128129)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:53.014647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:53.144587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8fc95d63834bd0a60930c36d128129'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:53.166139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8fc95d63834bd0a60930c36d128129'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:53.167616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8fc95d63834bd0a60930c36d128129'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:53.168186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8fc95d63834bd0a60930c36d128129'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:53.190435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8fc95d63834bd0a60930c36d128129'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:53.239538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 838/1648] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:58:52 2022] DELETE /identity/v3/users/0e8fc95d63834bd0a60930c36d128129 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:58:53.245090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:53.245657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:53.247647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:53.311707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:53.313114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:53.480790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 811/1649] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:53.641143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:53.642349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:53.818668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220423d2-060c-47f1-8a78-8b6829d3a5bf admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:53.847105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:53.857965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:53.862309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:53.877582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:53.920283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:53.922020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:53.968365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:53.972173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:54.001545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 839/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:58:54.006006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd775d6f-19a5-4d3b-8ebc-fb6252f0cc82 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:54.030653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:54.031747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:54.032168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:54.032598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] PATH_INFO: `/v3/projects/85b34d4bdbe24b7bb1cb9c4d8923da04` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:54.034203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b34d4bdbe24b7bb1cb9c4d8923da04)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:54.035417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: WARNING py.warnings [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:58:54.035417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: warnings.warn(msg) Sep 13 07:58:54.035417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: Sep 13 07:58:54.037221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:54.172157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b34d4bdbe24b7bb1cb9c4d8923da04', 'request_id': 'req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.191820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b34d4bdbe24b7bb1cb9c4d8923da04', 'request_id': 'req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.205451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b34d4bdbe24b7bb1cb9c4d8923da04', 'request_id': 'req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.281293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] The token cache is being invalidate because project 85b34d4bdbe24b7bb1cb9c4d8923da04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:58:54.281503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.283860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.289510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 812/1651] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:58:53 2022] DELETE /identity/v3/projects/85b34d4bdbe24b7bb1cb9c4d8923da04 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:54.293369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:54.320435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd98ca-3caa-4393-8d62-728eaa708d21 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:54.517012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:54.517670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:54.522830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:54.523054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] PATH_INFO: `/v3/users/55f857154b5d4d6aba2b0cdb657303fd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:54.524774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f857154b5d4d6aba2b0cdb657303fd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:54.529476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:54.544667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:54.655080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f857154b5d4d6aba2b0cdb657303fd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.680889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f857154b5d4d6aba2b0cdb657303fd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.681520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f857154b5d4d6aba2b0cdb657303fd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.682154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f857154b5d4d6aba2b0cdb657303fd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.709119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f857154b5d4d6aba2b0cdb657303fd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:54.766351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 840/1652] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:58:54 2022] DELETE /identity/v3/users/55f857154b5d4d6aba2b0cdb657303fd => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:55.991513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:55.993635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.024766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaccb6f-45c9-4639-a783-a095b2aa6ca7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.163229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:56.178860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.181367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.181577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.182616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.184300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.184724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] PATH_INFO: `/v3/projects/396240fee2a545858feb20fb12b58b19` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.186661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=396240fee2a545858feb20fb12b58b19)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.190047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.208571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95ef9f-e0d1-4fea-80e3-6afbbc2f6cf0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.226420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.227251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.227759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.228872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] PATH_INFO: `/v3/projects/e29a219bc4e74f4590fe5e44261e1821` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.236507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29a219bc4e74f4590fe5e44261e1821)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.239171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.348086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396240fee2a545858feb20fb12b58b19', 'request_id': 'req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.370510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396240fee2a545858feb20fb12b58b19', 'request_id': 'req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.390965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396240fee2a545858feb20fb12b58b19', 'request_id': 'req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.401644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29a219bc4e74f4590fe5e44261e1821', 'request_id': 'req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.430337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29a219bc4e74f4590fe5e44261e1821', 'request_id': 'req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.445118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29a219bc4e74f4590fe5e44261e1821', 'request_id': 'req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.448488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] The token cache is being invalidate because project 396240fee2a545858feb20fb12b58b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:58:56.448745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.452772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f9dd7db6-0131-4be9-ae39-dbabbe60fa6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.455935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 813/1653] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:58:55 2022] DELETE /identity/v3/projects/396240fee2a545858feb20fb12b58b19 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:56.461668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fbd4f-da84-41ca-ab39-28c3b9168086 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.461992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fbd4f-da84-41ca-ab39-28c3b9168086 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.462273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fbd4f-da84-41ca-ab39-28c3b9168086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.479845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] The token cache is being invalidate because project e29a219bc4e74f4590fe5e44261e1821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:58:56.480168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.480979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bfa1ba84-8d7e-4bfd-aa98-868755480c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:58:56.487452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 841/1654] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:58:56 2022] DELETE /identity/v3/projects/e29a219bc4e74f4590fe5e44261e1821 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:56.489426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed119-ef5f-4d75-9014-2399e26a422b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.489716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed119-ef5f-4d75-9014-2399e26a422b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.490020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed119-ef5f-4d75-9014-2399e26a422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.539146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-086fbd4f-da84-41ca-ab39-28c3b9168086 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:56.540390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-086fbd4f-da84-41ca-ab39-28c3b9168086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.549999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b2eed119-ef5f-4d75-9014-2399e26a422b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:56.553582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2eed119-ef5f-4d75-9014-2399e26a422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.680521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 814/1655] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:56.687703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.687989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.688284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.717057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 842/1656] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:56.720303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.720767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.721043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.730845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:56.734297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.749807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:56.759724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 815/1657] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:58:56.760873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.765838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:56.766991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:56.784757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3aa43-c2de-4e45-bafd-029e62a2bc59 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.798009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 843/1658] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:58:56.801759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.801759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.801759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.801759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.802064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:56.802064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.802064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.802712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.825500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3027a-a9c4-42c5-947e-0128e2558e0a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.836675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 816/1659] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:56.837250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.837771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.837894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.839034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.839431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.840108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:58:56.840451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.842192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.858952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ecdb-0aa1-4d11-9a03-462e111f8494 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.864224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.864579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.864862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.865137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.865861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.867933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.872775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 844/1660] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:58:56.874002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.898095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1b88b-1246-438f-b965-fbce966ab2fd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:56.910752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:56.911377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:56.911655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:56.911924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:56.912569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:56.914528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:56.942776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 817/1661] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:56.943752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:56.988203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 845/1662] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:58:56.991542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.047179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:57.096058 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea7b45-3d89-40a2-8311-6d8afb6e5960 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.165468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf408c4f-1825-4c38-8c8d-688fad1b681d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.179297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.185621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.185941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.187568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.189185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.192833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.275408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 846/1663] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/users => generated 282 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:58:57.277886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.291474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.292141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.292380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.294349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.295222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.299481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.312015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2279db-1368-4a31-b423-9f4c956610d7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.337270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.338229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.338723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.339136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.340067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.342434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.367949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 818/1664] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:56 2022] POST /identity/v3/users => generated 291 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:57.370965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.371322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 847/1665] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:57.372212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.393655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48dd0f-920a-46be-9b88-20abe07e9c66 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.394082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00b4e7-5e53-43ec-aff6-0504c71297fe admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.413191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.413864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.414146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.414416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.415244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.416151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.416788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.416885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.417064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.419790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.421120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.443020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.444471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 819/1666] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:57.444953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.478245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace247e-d148-410a-9299-9aef53c92f86 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.490042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 848/1667] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:57.491646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.499472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.502891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.502891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.502891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.502891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.503679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.524847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0700c4-799a-4f59-97b6-dfa5bff4f26a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.550072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.550072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.550072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.550072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] PATH_INFO: `/v3/projects/3895318b23c049a8a6e695e70efa2506/users/12edcd153e0344f6949be68a076ad42c/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.556723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 820/1668] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:57.557931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.576307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea994-7c1b-4b44-a2ae-f1bb02dbf040 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:57.592196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3895318b23c049a8a6e695e70efa2506, user_id=12edcd153e0344f6949be68a076ad42c, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.597673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c508a50-c497-4984-8dbb-ca1ad25b56d2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.601582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:57.611942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.612260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.612572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] PATH_INFO: `/v3/projects/13482a49bd8245c595d17787c1015522/users/a83ae891b5c34524bc162ff67d320ed1/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.637017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 849/1669] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:57 2022] PUT /identity/v3/projects/3895318b23c049a8a6e695e70efa2506/users/12edcd153e0344f6949be68a076ad42c/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:57.639762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.640325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.640808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.646538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13482a49bd8245c595d17787c1015522, user_id=a83ae891b5c34524bc162ff67d320ed1, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:57.650580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932b12-9628-45ff-bb4f-7f55ccdef33a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:57.677371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 821/1670] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:57 2022] PUT /identity/v3/projects/13482a49bd8245c595d17787c1015522/users/a83ae891b5c34524bc162ff67d320ed1/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:57.689938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:57.690231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:57.690508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:57.719950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] MFA Rules not processed for user `12edcd153e0344f6949be68a076ad42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:57.721690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:57.763997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] MFA Rules not processed for user `a83ae891b5c34524bc162ff67d320ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:57.765267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:57.904706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:57.930771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 850/1671] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:57.937489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:57.955666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 822/1672] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:57.956732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.021553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:58.093358 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2a12-3385-41a5-962a-cb414bd95f4b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.098345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.099819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.100105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.100848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.101054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.115299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.127988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5afeb-00c6-4eaa-ac77-75950d47c043 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.143225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.143881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.145912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.147403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.148217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.156989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.180538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 823/1673] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:57 2022] POST /identity/v3/projects => generated 381 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:58.185294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.234153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 851/1674] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:58:57 2022] POST /identity/v3/projects => generated 365 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:58.235092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.284786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04263b4-02b2-4ac5-8ce0-92e32d1b5e13 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.394387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968547b-e95a-493c-8996-731881ee1a82 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.405632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.406381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.407383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.407693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.408497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.410739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1c9e8-ef92-47d5-84d8-b5bf8e02f6ba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.421397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.422221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.422507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.422779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.423875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.430252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.445176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 852/1675] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:58 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:58:58.448271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.448641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.449500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.482751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 824/1676] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:58:58 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:58:58.483696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.511988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083de621-7443-4ec1-ae0a-bcaede0bbc67 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.526461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] MFA Rules not processed for user `0d7a4ca3d0904c1f8e3b9622fea8af8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:58.527704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:58.528460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.529087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.529363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.529630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.532715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.535822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.583731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 825/1677] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:58.584704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.596666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7b6c4-2808-473b-8fba-e50c741f4db9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.603001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.603555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.603829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.604102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.604751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.606686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.676403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 826/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:58.676403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.706211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a3191-3dd9-4425-8636-b115ec3cfce5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.714421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.715396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.716109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.718547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] PATH_INFO: `/v3/projects/e94ea2561fde469089753e13b409e054/users/c0e18323310c4ea4aebd07d8816c71f7/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.768756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 853/1679] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:58.779538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:58.788155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94ea2561fde469089753e13b409e054, user_id=c0e18323310c4ea4aebd07d8816c71f7, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.790207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:58.799769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b58a1-2df8-4519-8d1a-99fffc26ff9b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.810652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf7b8c-b4df-43aa-97ac-fd3482e59f50 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:58.823787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:58.830638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:58.831413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.834453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.834822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] PATH_INFO: `/v3/projects/528017949a394a6fabea6546b1598f41/users/c573c0bef5424c289c8f86a04016d015/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.852674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 827/1680] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:58 2022] PUT /identity/v3/projects/e94ea2561fde469089753e13b409e054/users/c0e18323310c4ea4aebd07d8816c71f7/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:58.858395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:58.858786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:58.858938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:58.913140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528017949a394a6fabea6546b1598f41, user_id=c573c0bef5424c289c8f86a04016d015, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:58.927049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:58.936228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] MFA Rules not processed for user `0d7a4ca3d0904c1f8e3b9622fea8af8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:58.940321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:58.964195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 854/1681] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:58 2022] PUT /identity/v3/projects/528017949a394a6fabea6546b1598f41/users/c573c0bef5424c289c8f86a04016d015/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:58.968500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.124224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 828/1682] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:59.125769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.170194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410013ce-6e9a-451a-a66e-3b52ed322f11 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:59.188151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:59.188562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.188842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.189252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.190136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:59.191773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:59.233783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 855/1683] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:59.237768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.263207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0cdcf-b389-4859-ba63-3c42a0e9ba3c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:59.276299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:59.278592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.278988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.279372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] PATH_INFO: `/v3/projects/e94ea2561fde469089753e13b409e054/users/c0e18323310c4ea4aebd07d8816c71f7/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.282405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94ea2561fde469089753e13b409e054, user_id=c0e18323310c4ea4aebd07d8816c71f7, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:59.312680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:59.348423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 856/1684] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:59 2022] PUT /identity/v3/projects/e94ea2561fde469089753e13b409e054/users/c0e18323310c4ea4aebd07d8816c71f7/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:59.353830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.358925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:59.391417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbe5fa-575b-44ff-8e5c-19e1f9d0f52a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:59.548242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:59.549997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.550463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.551169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.552007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:59.554960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2e840-f6d2-4384-acd2-d7e06ff4feaa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:59.571908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 829/1685] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:58:59.573987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.574840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.575922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.608786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe5722-ae6d-4027-b5ce-6c1af8d8ba00 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:59.630539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:59.633458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.634101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.634383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.658856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] MFA Rules not processed for user `c0e18323310c4ea4aebd07d8816c71f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:58:59.663114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:59.779035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:59.805146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:59.825076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:59.846797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:58:59.866300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 857/1686] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:58:59.867393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.873690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 830/1687] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:58:59.910894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25f1d0-7281-4b83-a595-b7d49b0d70e3 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:58:59.931512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:58:59.932570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.934062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.935108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] PATH_INFO: `/v3/projects/528017949a394a6fabea6546b1598f41/users/c573c0bef5424c289c8f86a04016d015/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:58:59.939333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:58:59.940235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528017949a394a6fabea6546b1598f41, user_id=c573c0bef5424c289c8f86a04016d015, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:58:59.941274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:58:59.943986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220369e-5652-4e40-9316-a7b27c29e618 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:58:59.983398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 858/1688] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:58:59 2022] PUT /identity/v3/projects/528017949a394a6fabea6546b1598f41/users/c573c0bef5424c289c8f86a04016d015/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:58:59.994203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:58:59.994203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:58:59.994685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.107845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] MFA Rules not processed for user `c573c0bef5424c289c8f86a04016d015`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:00.109083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:00.119953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee47bae-f3cd-46c6-a8df-f2978d221ce2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:00.280770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 859/1689] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:58:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:00.305035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:00.305671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:00.305985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:00.306257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.431025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:00.432313 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:00.443516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:00.445393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:00.467296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389758f9-69c1-4074-83ec-f1d0f9093dd8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:00.476497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:00.477142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:00.477382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 831/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:58:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:00.477477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:00.477680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.497275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:00.498765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650f84d-14f9-47bb-90d4-c9eb90a8be0a service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:00.515406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 860/1691] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:00.815162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:00.831088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea758e-05e4-468b-9085-bf729772ee56 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:00.835925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:00.836884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:00.837357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:00.837825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.938610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:00.960126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a899d-d588-4b38-a551-b8add96ede35 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:00.960413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a899d-d588-4b38-a551-b8add96ede35 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:00.960689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a899d-d588-4b38-a551-b8add96ede35 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.962419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 861/1692] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 07:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:00.974073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713305-c89e-4651-8656-7745c1856182 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:00.974349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713305-c89e-4651-8656-7745c1856182 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:00.974631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713305-c89e-4651-8656-7745c1856182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:00.986972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:00.989608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:01.021462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 832/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:01.056958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4d713305-c89e-4651-8656-7745c1856182 None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:01.060626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4d713305-c89e-4651-8656-7745c1856182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:01.278012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 862/1694] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:59:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:01.364708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:01.376604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:01.410846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010fc6-00ba-4145-a51c-60b8ca4514bb service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:01.423293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:01.423938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:01.424222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:01.424496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:01.451378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:01.453019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:01.475571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 833/1695] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:01.614383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:01.614933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:01.615270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:01.698574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] MFA Rules not processed for user `bada6717873d408a95058051f2589a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:01.700815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:01.805991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:01.852242 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 863/1696] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:01.868253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:01.869691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.018784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae809-5c59-42c1-99ac-1869aeac1bca service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:02.026687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:02.027654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.028217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.029278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.043144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:02.043144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a5afb-970c-4cc1-8e53-67c78052e259 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:02.044876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:02.058081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 834/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:02.116523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebeedb-1e94-4d80-b3d9-23ad47b3c303 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:02.122358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:02.123635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.123635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.123844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] PATH_INFO: `/v3/users/12edcd153e0344f6949be68a076ad42c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.125036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12edcd153e0344f6949be68a076ad42c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:02.127710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:02.187110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12edcd153e0344f6949be68a076ad42c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:02.200962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12edcd153e0344f6949be68a076ad42c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:02.202110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12edcd153e0344f6949be68a076ad42c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:02.204511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12edcd153e0344f6949be68a076ad42c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:02.218656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-30b3c4eb-e5fe-43f7-8bf8-267c526e2d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12edcd153e0344f6949be68a076ad42c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:02.251059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7ac0-dcb9-498c-b0f5-8b5e6f6e64fa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.251563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7ac0-dcb9-498c-b0f5-8b5e6f6e64fa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.252379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7ac0-dcb9-498c-b0f5-8b5e6f6e64fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.266492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 864/1698] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:02 2022] DELETE /identity/v3/users/12edcd153e0344f6949be68a076ad42c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:02.272568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89578-c73f-439b-9127-45800b42051f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.272891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89578-c73f-439b-9127-45800b42051f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.273349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89578-c73f-439b-9127-45800b42051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.292691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e58b7ac0-dcb9-498c-b0f5-8b5e6f6e64fa None None] MFA Rules not processed for user `272f5244f9724e00bd35dd09fedc6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:02.294325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e58b7ac0-dcb9-498c-b0f5-8b5e6f6e64fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.310248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-04b89578-c73f-439b-9127-45800b42051f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:02.311714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04b89578-c73f-439b-9127-45800b42051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.401668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 835/1699] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:02.404388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014180-d521-4104-b407-1aff0ca38097 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.404843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014180-d521-4104-b407-1aff0ca38097 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.405244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014180-d521-4104-b407-1aff0ca38097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.431332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 865/1700] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:02.445259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b89578-c73f-439b-9127-45800b42051f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:02.447296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-04b89578-c73f-439b-9127-45800b42051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.453365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:02.472038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-96014180-d521-4104-b407-1aff0ca38097 None None] MFA Rules not processed for user `272f5244f9724e00bd35dd09fedc6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:02.474841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96014180-d521-4104-b407-1aff0ca38097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.491209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 836/1701] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:02.516452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96014180-d521-4104-b407-1aff0ca38097 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:02.518251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96014180-d521-4104-b407-1aff0ca38097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.631556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:02.682245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b89578-c73f-439b-9127-45800b42051f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:02.691471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:02.692747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.692976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.693334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.713127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:02.717320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:02.721057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96014180-d521-4104-b407-1aff0ca38097 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:02.729990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:02.731475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.732210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.732581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.745013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:02.747940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:02.748139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 866/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:02.754790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:02.756843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:02.781205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 837/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:02.924312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc814e1b-8ade-4df2-b3fd-b3cebe4059ca service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:02.952852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:02.955274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:02.955541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:02.959447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:02.970388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:03.035297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:03.038755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:03.071793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 867/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:03.507295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:03.509128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:03.554330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c06bb-0478-4ede-b130-8c50b99970c7 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:03.565987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:03.568492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:03.568995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:03.569703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] PATH_INFO: `/v3/projects/3895318b23c049a8a6e695e70efa2506` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:03.572447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3895318b23c049a8a6e695e70efa2506)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:03.578385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:03.785930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3895318b23c049a8a6e695e70efa2506', 'request_id': 'req-45cafa6b-0298-45ba-aadf-f043990b1326'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:03.810461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3895318b23c049a8a6e695e70efa2506', 'request_id': 'req-45cafa6b-0298-45ba-aadf-f043990b1326'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:03.832048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3895318b23c049a8a6e695e70efa2506', 'request_id': 'req-45cafa6b-0298-45ba-aadf-f043990b1326'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:03.853411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] The token cache is being invalidate because project 3895318b23c049a8a6e695e70efa2506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:03.853708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:03.858731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:03.864925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 838/1705] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:03 2022] DELETE /identity/v3/projects/3895318b23c049a8a6e695e70efa2506 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:03.866140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:03.867172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:03.882535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:03.882698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cfda-c5d9-4a52-9c40-564ce270ae5f service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:04.018633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:04.019298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.019583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.019862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] PATH_INFO: `/v3/users/c0e18323310c4ea4aebd07d8816c71f7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.022415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e18323310c4ea4aebd07d8816c71f7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:04.024351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:04.115012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:04.133656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e18323310c4ea4aebd07d8816c71f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.134153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cafa6b-0298-45ba-aadf-f043990b1326 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:04.154239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:04.155762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.157536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e18323310c4ea4aebd07d8816c71f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.158226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e18323310c4ea4aebd07d8816c71f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.159200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e18323310c4ea4aebd07d8816c71f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.162912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.164572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] PATH_INFO: `/v3/users/afa6f32a8e9041429acb5ae15e01b6f1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.169645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa6f32a8e9041429acb5ae15e01b6f1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:04.171701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6dc47ed0-3bdc-49fd-8e21-8c7470c1ee60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e18323310c4ea4aebd07d8816c71f7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.175930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:04.223884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 868/1706] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:03 2022] DELETE /identity/v3/users/c0e18323310c4ea4aebd07d8816c71f7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:04.241309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eca1-27fb-4fdb-9bc6-2e8d93239afd None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.242752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eca1-27fb-4fdb-9bc6-2e8d93239afd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.243136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9eca1-27fb-4fdb-9bc6-2e8d93239afd None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.250816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 869/1707] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 07:59:04 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:04.256707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.256992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.257268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.303722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:04.307851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa6f32a8e9041429acb5ae15e01b6f1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.326937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:04.331105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa6f32a8e9041429acb5ae15e01b6f1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.331936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa6f32a8e9041429acb5ae15e01b6f1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.332750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:04.333366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa6f32a8e9041429acb5ae15e01b6f1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.356895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72e8843d-7345-4e14-8293-18f26eb3141d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa6f32a8e9041429acb5ae15e01b6f1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:04.425666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 839/1708] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:03 2022] DELETE /identity/v3/users/afa6f32a8e9041429acb5ae15e01b6f1 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:04.429970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.431256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.431714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.507441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 870/1709] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:59:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:04.513597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:04.514881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:04.565088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:04.567235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:04.607255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af945d8-69f7-4adb-acb5-df9b94dc2c28 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:04.725786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:04.726898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.727182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.727500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.740682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 840/1710] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:04.759520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:04.761944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:04.766547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:04.768089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:04.780462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 871/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:04.800280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c8311-cf71-408a-b461-7bf2d8e7962d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:04.808115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:04.809521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:04.810875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:04.811999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:04.830498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:04.832214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:04.848382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 841/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:05.587319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:05.589042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:05.621164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029d30d-42ab-4dda-b9fc-cf6d0c35e72c service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:05.634741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:05.635465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:05.635714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:05.636236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] PATH_INFO: `/v3/projects/9f884173ca424e9480348995d9288164` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:05.637974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f884173ca424e9480348995d9288164)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:05.639921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:05.807063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:05.808653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:05.808763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f884173ca424e9480348995d9288164', 'request_id': 'req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:05.822302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f884173ca424e9480348995d9288164', 'request_id': 'req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:05.840961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f884173ca424e9480348995d9288164', 'request_id': 'req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:05.842719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:05.864200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] The token cache is being invalidate because project 9f884173ca424e9480348995d9288164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:05.864453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:05.865144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fbb5218a-5f6b-4435-8963-6e0e79a82bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:05.869895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cff2d8-d95e-47a8-9726-c6c550806e52 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:05.872636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 872/1713] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:05 2022] DELETE /identity/v3/projects/9f884173ca424e9480348995d9288164 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:05.877724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d319a5-aaa2-4e0e-98e2-d6f95e325f31 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:05.878364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d319a5-aaa2-4e0e-98e2-d6f95e325f31 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:05.878650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d319a5-aaa2-4e0e-98e2-d6f95e325f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:05.964022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c7d319a5-aaa2-4e0e-98e2-d6f95e325f31 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:05.972785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c7d319a5-aaa2-4e0e-98e2-d6f95e325f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:06.186011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:06.186720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.187084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.187543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] PATH_INFO: `/v3/projects/e94ea2561fde469089753e13b409e054` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.193219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94ea2561fde469089753e13b409e054)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:06.202396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:06.265148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 873/1714] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:06.270560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.272229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.273782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.322200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:06.324202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:06.372752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 874/1715] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:06.384819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:06.427332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94ea2561fde469089753e13b409e054', 'request_id': 'req-e38d4cba-6ed8-413a-ba35-db005c0ee61f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:06.432907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:06.453790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88111f-738b-4c5e-9e7d-9822a607e098 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:06.459473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94ea2561fde469089753e13b409e054', 'request_id': 'req-e38d4cba-6ed8-413a-ba35-db005c0ee61f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:06.489945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94ea2561fde469089753e13b409e054', 'request_id': 'req-e38d4cba-6ed8-413a-ba35-db005c0ee61f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:06.527596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] The token cache is being invalidate because project e94ea2561fde469089753e13b409e054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:06.528886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:06.530082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e38d4cba-6ed8-413a-ba35-db005c0ee61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:06.534821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 842/1716] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:05 2022] DELETE /identity/v3/projects/e94ea2561fde469089753e13b409e054 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:06.540023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8eaa-be09-45fd-aed2-6d95a8d785eb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.540023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8eaa-be09-45fd-aed2-6d95a8d785eb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.540023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8eaa-be09-45fd-aed2-6d95a8d785eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.623243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-dafd8eaa-be09-45fd-aed2-6d95a8d785eb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:06.627544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dafd8eaa-be09-45fd-aed2-6d95a8d785eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:06.719160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:06.721331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.721683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.722312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.723540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:06.724024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:06.725817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:06.770749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 875/1717] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:06.772023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:06.773801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:06.805010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc108cc-920a-4fef-838b-12cce02ef7e4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:06.829767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:06.830500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.830799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.831071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.831901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:06.834575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:06.876037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 843/1718] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:06.878511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:06.881433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:06.881955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:06.910639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:06.913029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:06.927813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 844/1719] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:06.932221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:06.949341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 876/1720] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:06 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:06.954098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:06.955483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10938f7-515b-4a6c-9567-7902fe020550 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.050153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.051050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.051409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.051750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.052479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:07.052872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.055350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.077572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 845/1721] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:07.083901 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:07.086233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.095107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e26768-3a57-4ba8-a149-03fe7d31a4bb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.103375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd553-db93-4d0b-baaa-51439213d343 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.108479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.109112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.109390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.109707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.110372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.112304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.121281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.122604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.122935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.123165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.123882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.125914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7813b3-7c88-4fac-83f1-882ca52f3727 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.182018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 846/1722] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:07 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:07.183315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.183646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.183861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.190024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 877/1723] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:06 2022] POST /identity/v3/users => generated 284 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:07.190925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.218835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261277-b1f0-4c0b-9f0e-a8f05b2fdcff admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.259193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] MFA Rules not processed for user `272f5244f9724e00bd35dd09fedc6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:07.260295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:07.319677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.320289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.320644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.321326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.322004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.324182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.331896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:07.357684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 847/1724] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:07.358733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.371476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 878/1725] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:07 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:07.373169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.376035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:07.382120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573fb1-7bb6-4e08-85a5-88f78b38b80d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.390573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e2597-1524-4972-af06-46cc3bf55475 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.396146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.396895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.397267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.397724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] PATH_INFO: `/v3/users/49bd70d7364849a68ae503d7ba6cc9f1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.397966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.398868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.399288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49bd70d7364849a68ae503d7ba6cc9f1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.399451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.399853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.401003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.401569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.403626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309e7d9-bf62-41ed-88b7-61cf1bbd390b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.416324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 848/1726] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:07.417865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.418151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.418973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.442817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] MFA Rules not processed for user `bada6717873d408a95058051f2589a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:07.444153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:07.541155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49bd70d7364849a68ae503d7ba6cc9f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:07.562956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49bd70d7364849a68ae503d7ba6cc9f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:07.563931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49bd70d7364849a68ae503d7ba6cc9f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:07.564783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49bd70d7364849a68ae503d7ba6cc9f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:07.619239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 849/1727] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:07.619239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.619550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49bd70d7364849a68ae503d7ba6cc9f1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:07.620484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:07.639147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff07341-4435-4876-a0a0-594bd5dbaf9b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.644623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.645456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.645868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.646228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.647470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.648999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.683616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 850/1728] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:07.685036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.692106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:07.704250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 879/1729] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:07 2022] DELETE /identity/v3/users/49bd70d7364849a68ae503d7ba6cc9f1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:07.705541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.717143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac2ff-6d3a-4228-ad5c-1de1875bdca4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.850072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:07.889920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d9e5d-1131-42b6-a8c7-24f06d382c06 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.907622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.908160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.908386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.908483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.908560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.908816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.908912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.909144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.909819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.912875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceded3f-8a6d-4ee0-b986-461bbbd89870 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.917657 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:07.920472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:07.937796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 851/1730] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:07.943780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.943780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.943780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.954913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 880/1731] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:07.955849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:07.968787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46bf65-87e3-4b5d-b3b4-5e924ae696bd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:07.974015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:07.974701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:07.975169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:07.975453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] PATH_INFO: `/v3/projects/267d5593ae454d15853d8410e9bf4cc4/users/ad2893fdd2334196881ab7640153cfbc/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:07.984648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:07.986020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:08.000021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 852/1732] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:08.001093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:08.004560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267d5593ae454d15853d8410e9bf4cc4, user_id=ad2893fdd2334196881ab7640153cfbc, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:08.007306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:08.031212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b13eb-aef8-4150-b6c8-58d81a492de2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:08.032465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 881/1733] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:07 2022] PUT /identity/v3/projects/267d5593ae454d15853d8410e9bf4cc4/users/ad2893fdd2334196881ab7640153cfbc/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:08.033631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:08.035429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:08.166102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3839b-1f19-4ceb-a38b-6053f41d0ce5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:08.179679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:08.180371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.180477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:08.180667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.180927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] PATH_INFO: `/v3/projects/17723d910e304bc2b483c2e72dee6bdb/users/018ab1d00700418f9290440ed702dd9d/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.185302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.185760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.186274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.190255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:08.216115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:08.217154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17723d910e304bc2b483c2e72dee6bdb, user_id=018ab1d00700418f9290440ed702dd9d, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:08.219300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766e532-3dd9-4a03-91fa-319251c99a22 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:08.220047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205d8b0-fa9f-4f6d-ab62-59ddf1583409 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:08.243501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 882/1734] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:08.248452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.249787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.251475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.256335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 853/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:08 2022] PUT /identity/v3/projects/17723d910e304bc2b483c2e72dee6bdb/users/018ab1d00700418f9290440ed702dd9d/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:08.267432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.267791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.268101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.349152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] MFA Rules not processed for user `ad2893fdd2334196881ab7640153cfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:08.350907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:08.351039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] MFA Rules not processed for user `018ab1d00700418f9290440ed702dd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:08.352221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:08.484161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 883/1736] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:08.489606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:08.534089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 854/1737] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:08.535335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:08.650758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:08.688498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604cd45-430b-4429-8a5c-ba8f7d91efa2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:08.702969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:08.705715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.706243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.706531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.711859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:08.719269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:08.752212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bfabd-a690-423b-a64a-ab797f8e9083 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:08.771721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:08.772472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:08.772673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:08.776330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:08.779399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:08.783670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:08.812810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 884/1738] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/projects => generated 369 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:08.815066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:08.865210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 855/1739] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/projects => generated 365 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:08.866363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.007471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e155d-e97d-4e92-b9fe-7bfd99d05120 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.022377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed3578-8e04-4385-b516-7cf785224665 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.026024 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.027171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.028059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.028687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.029163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.029449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.030347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.032819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.035875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.036216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.036909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.039259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.042095 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:09.084115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 856/1740] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:09.095079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.130184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 885/1741] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:08 2022] POST /identity/v3/users => generated 291 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:09.133117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.133772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d6953-94e0-4f63-9e88-8f805184a58f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.151394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.153139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.156857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.158052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.159615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.166465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.168952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ac2c2-5b96-4596-80eb-8cb3b7256d4c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.191078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.198646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.198646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.198894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.200888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.202377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.214155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 857/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:09.215710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.236558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 886/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:09.240779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.241188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee278b39-59a5-413a-ae8e-f3d42171075f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.259012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.259732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.260038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.260336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] PATH_INFO: `/v3/projects/c623c591cae549339b3a0c1909752f3d/users/b3a28068ed8442a189c6dde684498da1/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.268149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13560b9b-7868-4ca3-a65d-1b9597aae626 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.289852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.310419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.310739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.311952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] PATH_INFO: `/v3/projects/b50c5581c26446bab76b98797e021051/users/4f723b82c63d48a8b16ac63f9f125262/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.331967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c623c591cae549339b3a0c1909752f3d, user_id=b3a28068ed8442a189c6dde684498da1, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.351243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.403482 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50c5581c26446bab76b98797e021051, user_id=4f723b82c63d48a8b16ac63f9f125262, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.411588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.440562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 858/1744] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:09 2022] PUT /identity/v3/projects/c623c591cae549339b3a0c1909752f3d/users/b3a28068ed8442a189c6dde684498da1/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:09.458467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.467869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 887/1745] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:09 2022] PUT /identity/v3/projects/b50c5581c26446bab76b98797e021051/users/4f723b82c63d48a8b16ac63f9f125262/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:09.469692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.671900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177adfd1-fa60-4c5e-b9af-35908ad393cc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.693285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.693955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.694239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.694617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.695284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.697190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b487b61-6f8a-4946-8350-dbb166067668 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.728261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 888/1746] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:09.750467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e56eb-424c-499a-b6b1-b56bbb7c90be None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.751961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e56eb-424c-499a-b6b1-b56bbb7c90be None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.752394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e56eb-424c-499a-b6b1-b56bbb7c90be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.758976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19521be-bac2-4462-8635-b5317eceb3ed admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.779322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.784626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.786767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.787157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.788832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.792492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.831784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 859/1747] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:09.833343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:09.853452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c426677-5400-4def-885d-475e3f9b9d92 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:09.869555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:09.870631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:09.871132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:09.871649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] PATH_INFO: `/v3/projects/b50c5581c26446bab76b98797e021051/users/4f723b82c63d48a8b16ac63f9f125262/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:09.877051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50c5581c26446bab76b98797e021051, user_id=4f723b82c63d48a8b16ac63f9f125262, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:09.879996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:09.886267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:09.911415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-038e56eb-424c-499a-b6b1-b56bbb7c90be None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:09.917659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-038e56eb-424c-499a-b6b1-b56bbb7c90be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:09.943598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 860/1748] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:09 2022] PUT /identity/v3/projects/b50c5581c26446bab76b98797e021051/users/4f723b82c63d48a8b16ac63f9f125262/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:09.943598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:10.165334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 889/1749] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 07:59:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:10.166383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba77f-0e63-44a5-a66f-de6fab121491 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:10.166684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba77f-0e63-44a5-a66f-de6fab121491 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:10.166963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cba77f-0e63-44a5-a66f-de6fab121491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:10.174868 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa30de1-694b-4437-bdeb-70217c7ab7f4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:10.199976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:10.217836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:10.220239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:10.220558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] PATH_INFO: `/v3/projects/c623c591cae549339b3a0c1909752f3d/users/b3a28068ed8442a189c6dde684498da1/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:10.231877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c623c591cae549339b3a0c1909752f3d, user_id=b3a28068ed8442a189c6dde684498da1, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:10.264014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:10.308642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-11cba77f-0e63-44a5-a66f-de6fab121491 None None] MFA Rules not processed for user `4f723b82c63d48a8b16ac63f9f125262`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:10.310872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-11cba77f-0e63-44a5-a66f-de6fab121491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:10.328608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 861/1750] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:09 2022] PUT /identity/v3/projects/c623c591cae549339b3a0c1909752f3d/users/b3a28068ed8442a189c6dde684498da1/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:10.331749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:10.339042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:10.439985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:10.521629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11910f-c448-4bf3-abb5-7ae672504ab3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:10.533350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 890/1751] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:10.533350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:10.533350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:10.533879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:10.534607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:10.535351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:10.535745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:10.536128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] PATH_INFO: `/v3/projects/27168502db1845468d219bbdcf68f42b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:10.538609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27168502db1845468d219bbdcf68f42b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:10.541170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:10.605345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] MFA Rules not processed for user `b3a28068ed8442a189c6dde684498da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:10.607517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:10.676987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27168502db1845468d219bbdcf68f42b', 'request_id': 'req-00decc7f-823d-44ff-a0f1-0d7b8b21619b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:10.692961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27168502db1845468d219bbdcf68f42b', 'request_id': 'req-00decc7f-823d-44ff-a0f1-0d7b8b21619b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:10.710143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27168502db1845468d219bbdcf68f42b', 'request_id': 'req-00decc7f-823d-44ff-a0f1-0d7b8b21619b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:10.735799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] The token cache is being invalidate because project 27168502db1845468d219bbdcf68f42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:10.738380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:10.739610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:10.748201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 862/1752] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:10 2022] DELETE /identity/v3/projects/27168502db1845468d219bbdcf68f42b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:10.748201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:10.751334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:10.788402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 891/1753] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:10.789061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:10.796086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:10.827921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:10.929796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d8ed5-e26e-49d3-9964-f6df2db0fc82 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.090380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.091051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.091464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.092012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.094884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.114883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00decc7f-823d-44ff-a0f1-0d7b8b21619b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.139901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.140546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.140891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.141162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.143210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.253205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.254728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bfeff-877d-48f5-a3f3-96375a7553c6 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:11.275882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:11.276537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 892/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:11.281090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd315-f7aa-4601-931e-c54723037cef None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.281381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd315-f7aa-4601-931e-c54723037cef None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.281794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd315-f7aa-4601-931e-c54723037cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.313536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-94bbd315-f7aa-4601-931e-c54723037cef None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:11.314897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-94bbd315-f7aa-4601-931e-c54723037cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.322708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.324883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:11.356278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 863/1755] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 07:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:59:11.357189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:11.358878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.411065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586eb6-dd58-448a-8995-aba18ec39e8f service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.440014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.440014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.440645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.440984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.584634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 893/1756] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:11.593507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.593832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.594180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.666386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.671354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:11.675115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:11.676384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.712035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 864/1757] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:11.717038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:11.725129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 894/1758] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:11.729927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:11.739941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fc34-becc-458e-b8fc-24f879a42769 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.753840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.754542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.754927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.755301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.761075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:11.797038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d7816-d474-48ae-8199-60d5b258e087 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.809096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.809928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.810210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.810491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.811225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:11.811540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.813243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:11.820508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:11.841580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 895/1759] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:11.846418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:11.904348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306417d-495d-489f-a460-4204c03daf01 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:11.924878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.939583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:11.940208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:11.940487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:11.940760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:11.941393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:11.943311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:11.951668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:12.030904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 896/1760] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:11 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:12.035071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 865/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:12.038956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.238443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a4393-7860-40ce-9a79-d79e3dde08e7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:12.248033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:12.248684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.248965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.249259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.259327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:12.261835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:12.317710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 897/1762] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:12 2022] POST /identity/v3/users => generated 303 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:12.322467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.380422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8cec-4ac7-472f-83dc-954006c09be9 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:12.398462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:12.405523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.406709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.406933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.407716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:12.414803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:12.447888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 866/1763] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:12.452366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.472497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7517e-c821-4d75-9209-a90fff9246bc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:12.479623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:12.480495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.481506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.482215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.483021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:12.485087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c3733-90e8-479b-8609-2e2ccab1ebb4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:12.512847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 898/1764] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:12.514415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.526254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01565-0acc-4a9f-853c-9c22db114781 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:12.532660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:12.533631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.534136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.534650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] PATH_INFO: `/v3/projects/28b867f93c7741519684967090242468/users/6d056384262444cf903eb876987f3b18/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.555766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b867f93c7741519684967090242468, user_id=6d056384262444cf903eb876987f3b18, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:12.562394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:12.604898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 867/1765] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:12 2022] PUT /identity/v3/projects/28b867f93c7741519684967090242468/users/6d056384262444cf903eb876987f3b18/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:12.607187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.607991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.608387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.622566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.652414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:12.656038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:12.861955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:12.881438 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532b91-3466-4dfe-9f3a-c75ac5c23364 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:12.892761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:12.894445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:12.906118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 899/1766] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:12.908959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:12.909397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:12.912755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:12.963358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:12.965387 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:13.116325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:13.119202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:13.136259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926bb5e-120e-443b-8cc0-cba4dd8efacc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:13.146208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:13.148474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:13.148938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:13.153030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:13.164303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 868/1767] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:13.166621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:13.172907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:13.216531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:13.221543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:13.226756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc7efd-cb2a-4385-a00c-89838fe91e7f service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:13.270383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3396d-1c8b-473c-81ad-6b655224dd38 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:13.274116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 900/1768] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:13.294781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:13.297297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:13.298061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:13.298479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] PATH_INFO: `/v3/users/53108e570b2c4d1bbd028c59b64b496b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:13.302909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=53108e570b2c4d1bbd028c59b64b496b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:13.307725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:13.407297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53108e570b2c4d1bbd028c59b64b496b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:13.444842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53108e570b2c4d1bbd028c59b64b496b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:13.445518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53108e570b2c4d1bbd028c59b64b496b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:13.446551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53108e570b2c4d1bbd028c59b64b496b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:13.481362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-da44e701-6c56-4be2-8279-d9fddf336c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53108e570b2c4d1bbd028c59b64b496b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:13.584020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 869/1769] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:13 2022] DELETE /identity/v3/users/53108e570b2c4d1bbd028c59b64b496b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:13.593111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:13.593542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:13.593965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:13.665620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:13.668071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:13.792300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:13.792684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:13.793043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:13.854679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:13.859366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:13.881214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 901/1770] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:13.959110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:13.960725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:14.073205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 870/1771] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 07:59:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:14.087867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:14.089390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:14.117196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8f1ed-0781-4033-8137-54937016d9b8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:14.135072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:14.135644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:14.135928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:14.136555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:14.170027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:14.177554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:14.185240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62beda9b-c7ef-45c5-a5e5-57c0f3dd64b9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:14.197269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:14.198531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:14.198820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:14.199094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:14.205332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 871/1772] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:14.267081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:14.268841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbda82a-4f77-4f9e-9e01-154ba447a903 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:14.343838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 902/1773] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:14.709414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:14.710986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:14.727526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:14.742089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f13f3-c8e6-4cb8-9967-7e4dc2fe5eca service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:14.760606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:14.761415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:14.761864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:14.762264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] PATH_INFO: `/v3/users/a83ae891b5c34524bc162ff67d320ed1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:14.765444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83ae891b5c34524bc162ff67d320ed1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:14.768088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:14.940741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83ae891b5c34524bc162ff67d320ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:14.954307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83ae891b5c34524bc162ff67d320ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:14.955587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83ae891b5c34524bc162ff67d320ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:14.956834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83ae891b5c34524bc162ff67d320ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:14.983849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83ae891b5c34524bc162ff67d320ed1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:15.062509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 872/1774] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:14 2022] DELETE /identity/v3/users/a83ae891b5c34524bc162ff67d320ed1 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:15.067304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:15.067842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:15.068312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:15.111101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:15.113538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:15.333255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 903/1775] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:15.365628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:15.367193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:15.558191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0112a-4885-4f66-980d-91c48f3bb0bf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:15.568901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:15.569624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:15.569879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:15.575332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:15.626624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:15.631549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5ca84-32f7-45c4-90a4-820296a422d9 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:15.677373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 873/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:16.620044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:16.622085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:16.669557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e79f8-598f-45b8-af82-f87030cda341 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:16.693222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:16.695418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:16.695801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:16.698692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] PATH_INFO: `/v3/users/ad2893fdd2334196881ab7640153cfbc` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:16.704729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2893fdd2334196881ab7640153cfbc)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:16.714355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:16.841202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:16.888965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2893fdd2334196881ab7640153cfbc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:16.927146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2893fdd2334196881ab7640153cfbc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:16.928094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2893fdd2334196881ab7640153cfbc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:16.930141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2893fdd2334196881ab7640153cfbc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:16.954416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2893fdd2334196881ab7640153cfbc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.025403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:17.026044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:17.026044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:17.036625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 904/1777] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:16 2022] DELETE /identity/v3/users/ad2893fdd2334196881ab7640153cfbc => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:17.081825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:17.085420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:17.142956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:17.144745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:17.282974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 874/1778] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:17.389174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:17.391144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:17.459028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:17.497758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9b69b-6cde-48f9-8921-40b1d61376ba admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:17.505866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:17.506613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:17.507403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:17.508304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] PATH_INFO: `/v3/users/018ab1d00700418f9290440ed702dd9d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:17.509936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018ab1d00700418f9290440ed702dd9d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:17.512961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:17.584758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801ffa-d08a-40c8-ad51-427223cb803a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:17.591721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:17.592452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:17.592740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:17.593160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:17.619755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:17.628268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c4c0a-6739-458f-8688-6106e52270ec service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:17.649424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018ab1d00700418f9290440ed702dd9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.659196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 875/1779] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:17.666009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018ab1d00700418f9290440ed702dd9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.666866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018ab1d00700418f9290440ed702dd9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.668735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018ab1d00700418f9290440ed702dd9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.690610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018ab1d00700418f9290440ed702dd9d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:17.727364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 905/1780] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:17 2022] DELETE /identity/v3/users/018ab1d00700418f9290440ed702dd9d => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:17.735569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:17.735964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:17.736308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:17.771454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:17.773506 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:17.946581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 876/1781] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:18.019508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:18.021319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:18.103608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:18.144819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:18.147365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:18.153850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a04c10-95bf-4e0e-b9eb-a4c3741806f1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:18.166224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:18.167341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:18.168959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8957ca-ba3a-4e4e-a45e-1e0c0f693aaf None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:18.169541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:18.169955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:18.186019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:18.187878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:18.188175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:18.188499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] PATH_INFO: `/v3/projects/13482a49bd8245c595d17787c1015522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:18.193096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13482a49bd8245c595d17787c1015522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:18.196483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:18.198631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:18.205949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:18.295925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 906/1782] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:18.329284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13482a49bd8245c595d17787c1015522', 'request_id': 'req-dac820a5-49b7-44d9-84c9-37962361a21c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.362136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13482a49bd8245c595d17787c1015522', 'request_id': 'req-dac820a5-49b7-44d9-84c9-37962361a21c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.400016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13482a49bd8245c595d17787c1015522', 'request_id': 'req-dac820a5-49b7-44d9-84c9-37962361a21c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.440933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] The token cache is being invalidate because project 13482a49bd8245c595d17787c1015522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:18.441155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.441882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-dac820a5-49b7-44d9-84c9-37962361a21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.446905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 877/1783] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:18 2022] DELETE /identity/v3/projects/13482a49bd8245c595d17787c1015522 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:18.448525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:18.466410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ce89-ab18-4afc-b7bc-2264a8f59948 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:18.522891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:18.523316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:18.523445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:18.528910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 878/1784] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:59:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:18.633726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:18.635690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:18.724402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:18.725066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:18.725348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:18.725623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] PATH_INFO: `/v3/users/c573c0bef5424c289c8f86a04016d015` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:18.729993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c573c0bef5424c289c8f86a04016d015)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:18.739946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:18.765203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:18.821893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d5c4-27a4-440e-8611-53708403690a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:18.838373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:18.839222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:18.840401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:18.844670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] PATH_INFO: `/v3/projects/97efdf3e4d9d4aab9760afae95a6ddcf` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:18.846870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97efdf3e4d9d4aab9760afae95a6ddcf)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:18.869559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:18.903580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c573c0bef5424c289c8f86a04016d015'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.915526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c573c0bef5424c289c8f86a04016d015'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.919869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c573c0bef5424c289c8f86a04016d015'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.921437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c573c0bef5424c289c8f86a04016d015'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:18.958319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3d4246f0-b826-4f08-abd3-a50b5ffe0ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c573c0bef5424c289c8f86a04016d015'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.018226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 907/1785] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:18 2022] DELETE /identity/v3/users/c573c0bef5424c289c8f86a04016d015 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:19.025691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139187-6095-4a1e-b959-72f7d8b6ddad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:19.028821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139187-6095-4a1e-b959-72f7d8b6ddad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:19.030179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d139187-6095-4a1e-b959-72f7d8b6ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:19.088757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97efdf3e4d9d4aab9760afae95a6ddcf', 'request_id': 'req-45820542-cc27-4822-8d6e-6c0c6f9bfa10'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.098555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2d139187-6095-4a1e-b959-72f7d8b6ddad None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:19.100233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2d139187-6095-4a1e-b959-72f7d8b6ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:19.102415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97efdf3e4d9d4aab9760afae95a6ddcf', 'request_id': 'req-45820542-cc27-4822-8d6e-6c0c6f9bfa10'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.121714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97efdf3e4d9d4aab9760afae95a6ddcf', 'request_id': 'req-45820542-cc27-4822-8d6e-6c0c6f9bfa10'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.137774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] The token cache is being invalidate because project 97efdf3e4d9d4aab9760afae95a6ddcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:19.138170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.143346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.145390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 879/1786] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:18 2022] DELETE /identity/v3/projects/97efdf3e4d9d4aab9760afae95a6ddcf => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:19.149492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:19.151039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:19.232856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:19.305224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45820542-cc27-4822-8d6e-6c0c6f9bfa10 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:19.315018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:19.316534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:19.317361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:19.318305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] PATH_INFO: `/v3/users/988bb7e503854780a95ba6919e7b3853` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:19.320772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=988bb7e503854780a95ba6919e7b3853)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:19.326601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:19.358839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 908/1787] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:59:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:19.554755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988bb7e503854780a95ba6919e7b3853'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.575227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988bb7e503854780a95ba6919e7b3853'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.580992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988bb7e503854780a95ba6919e7b3853'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.583817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988bb7e503854780a95ba6919e7b3853'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.614181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988bb7e503854780a95ba6919e7b3853'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:19.651431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:19.670178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 880/1788] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:19 2022] DELETE /identity/v3/users/988bb7e503854780a95ba6919e7b3853 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:20.144339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.145348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.145630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:20.197160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] MFA Rules not processed for user `6d056384262444cf903eb876987f3b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:20.198540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:20.259229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:20.261380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:20.408573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 909/1789] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:20.411400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:20.414419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:20.470099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e93e-c0cb-40a5-a4f8-1c276a867b7c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:20.493975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:20.493975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.493975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.493975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:20.494276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:20.532116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:20.533862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:20.576350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 881/1790] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:20.577573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d7ae-1680-4894-9759-880aab697b7b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:20.584591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:20.586500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.586902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.588110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:20.590897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:20.592998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:20.660278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 910/1791] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:20 2022] POST /identity/v3/projects => generated 405 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:20.661633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:20.679087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be45f7-518b-4348-8a19-6610f87592fb service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:20.804061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:20.805786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.806280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.806912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:20.808141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:20.810876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:20.855754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 882/1792] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:20 2022] POST /identity/v3/users => generated 309 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:20.857478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:20.873721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713d9b7-d816-4906-a10f-d869900a87d3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:20.890649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:20.892687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.893887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.894255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:20.897894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:20.899271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:20.941938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 911/1793] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:20.944205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:20.963772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a52893-9c77-4e06-a65a-1ab6884b8c9b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:20.972890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:20.976068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:20.976593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:20.977085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] PATH_INFO: `/v3/projects/778e78e5e3364655931f1bc13d69a7cc/users/309e981f1d944bc487e0d54b5a582ce6/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.034078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e78e5e3364655931f1bc13d69a7cc, user_id=309e981f1d944bc487e0d54b5a582ce6, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.039714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.063870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 883/1794] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:20 2022] PUT /identity/v3/projects/778e78e5e3364655931f1bc13d69a7cc/users/309e981f1d944bc487e0d54b5a582ce6/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:21.067407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.084717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72410c75-ec5e-477c-8e77-090ccce752e5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:21.109841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.112082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:21.201048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:21.239200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:21.273920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:21.273920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.273920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.273920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.285143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.285143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d4ee7-26a9-4286-be14-f9d62ce44489 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.314598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 912/1795] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:21.316767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.317059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.320512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.325508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac161d-c202-4201-89ee-e86658f28df8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:21.326056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 913/1796] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:59:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:21.326798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.329260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:21.343842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:21.344680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.345263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.345638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] PATH_INFO: `/v3/projects/267d5593ae454d15853d8410e9bf4cc4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.367544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc758a-2c8d-421c-8bf5-1ff290616f30 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:21.372805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:21.373467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.373766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.374069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] PATH_INFO: `/v3/projects/17723d910e304bc2b483c2e72dee6bdb` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.375314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17723d910e304bc2b483c2e72dee6bdb)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.377673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.386179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] RBAC: Authorizing `identity:delete_project(project_id=267d5593ae454d15853d8410e9bf4cc4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.392257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.486622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17723d910e304bc2b483c2e72dee6bdb', 'request_id': 'req-a3351e44-1c2c-461b-98e0-9475c9e4df60'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.507176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17723d910e304bc2b483c2e72dee6bdb', 'request_id': 'req-a3351e44-1c2c-461b-98e0-9475c9e4df60'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.524585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17723d910e304bc2b483c2e72dee6bdb', 'request_id': 'req-a3351e44-1c2c-461b-98e0-9475c9e4df60'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.529889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267d5593ae454d15853d8410e9bf4cc4', 'request_id': 'req-216158e9-ab20-4872-b563-ccb666faf12a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.550876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267d5593ae454d15853d8410e9bf4cc4', 'request_id': 'req-216158e9-ab20-4872-b563-ccb666faf12a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.558092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] The token cache is being invalidate because project 17723d910e304bc2b483c2e72dee6bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:21.558604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.559505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.562187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 914/1797] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:21 2022] DELETE /identity/v3/projects/17723d910e304bc2b483c2e72dee6bdb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:21.563359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.564931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267d5593ae454d15853d8410e9bf4cc4', 'request_id': 'req-216158e9-ab20-4872-b563-ccb666faf12a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.565170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:21.588890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] The token cache is being invalidate because project 267d5593ae454d15853d8410e9bf4cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:21.589256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.590455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.595037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 884/1798] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:21 2022] DELETE /identity/v3/projects/267d5593ae454d15853d8410e9bf4cc4 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:21.595972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.607383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:21.770854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3351e44-1c2c-461b-98e0-9475c9e4df60 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:21.802513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216158e9-ab20-4872-b563-ccb666faf12a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:21.819857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:21.820567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.824839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.826619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] PATH_INFO: `/v3/users/b3a28068ed8442a189c6dde684498da1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.831518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a28068ed8442a189c6dde684498da1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.837760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.929444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:21.930449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:21.930902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:21.931428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] PATH_INFO: `/v3/projects/778e78e5e3364655931f1bc13d69a7cc/users/309e981f1d944bc487e0d54b5a582ce6/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:21.935050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e78e5e3364655931f1bc13d69a7cc, user_id=309e981f1d944bc487e0d54b5a582ce6, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:21.938414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:21.953418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a28068ed8442a189c6dde684498da1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.962665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 915/1799] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:21 2022] PUT /identity/v3/projects/778e78e5e3364655931f1bc13d69a7cc/users/309e981f1d944bc487e0d54b5a582ce6/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:21.964939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:21.965553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a28068ed8442a189c6dde684498da1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.967254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a28068ed8442a189c6dde684498da1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.967638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:21.968290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a28068ed8442a189c6dde684498da1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.981830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e3d9c7dd-92b1-41f9-aba2-59b740f26e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a28068ed8442a189c6dde684498da1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:21.984254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:22.011564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:22.036990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 885/1800] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:21 2022] DELETE /identity/v3/users/b3a28068ed8442a189c6dde684498da1 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:22.036990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.038949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.042319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.109029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:22.112400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:22.155690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488bd6a-3878-4355-8611-ce3cc001923d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:22.323426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:22.326982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.328572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.330300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] PATH_INFO: `/v3/users/4f723b82c63d48a8b16ac63f9f125262` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.334076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 886/1801] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:59:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:22.335080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f723b82c63d48a8b16ac63f9f125262)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:22.340312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:22.352752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:22.354664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:22.380246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30442c9e-ba0b-4232-9c82-cf46a11486d8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:22.396594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:22.401196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.401786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.402158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.424986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:22.430249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:22.477471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 887/1802] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:22.535530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f723b82c63d48a8b16ac63f9f125262'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.551836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:22.553382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:22.568777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f723b82c63d48a8b16ac63f9f125262'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.570907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f723b82c63d48a8b16ac63f9f125262'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.574837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f723b82c63d48a8b16ac63f9f125262'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.577454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665406d-1392-4e8b-af55-74bf95913190 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:22.584480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:22.585206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.585476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.585981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] PATH_INFO: `/v3/projects/528017949a394a6fabea6546b1598f41` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.587237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=528017949a394a6fabea6546b1598f41)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:22.589262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:22.618726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b93f8204-2be8-4cef-ac44-0e6b5ab1e6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f723b82c63d48a8b16ac63f9f125262'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.658434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 916/1803] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:21 2022] DELETE /identity/v3/users/4f723b82c63d48a8b16ac63f9f125262 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:22.696646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528017949a394a6fabea6546b1598f41', 'request_id': 'req-228b3afc-d1ce-47e3-9154-5ac4d10673fa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.709280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528017949a394a6fabea6546b1598f41', 'request_id': 'req-228b3afc-d1ce-47e3-9154-5ac4d10673fa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.722695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528017949a394a6fabea6546b1598f41', 'request_id': 'req-228b3afc-d1ce-47e3-9154-5ac4d10673fa'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.740828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] The token cache is being invalidate because project 528017949a394a6fabea6546b1598f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:22.741371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.742185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-228b3afc-d1ce-47e3-9154-5ac4d10673fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:22.744031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 888/1804] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:22 2022] DELETE /identity/v3/projects/528017949a394a6fabea6546b1598f41 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:22.752084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.752369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.752649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.843118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:22.844408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:22.931196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:22.989550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:22.989671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:22.989999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:22.991473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 917/1805] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:23.006057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.010051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.028240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:23.029554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.047859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04680f-0c2f-4513-b5e2-fe28cfa63431 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.054580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 889/1806] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:23.055579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.066803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.069373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.070772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.072363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] PATH_INFO: `/v3/projects/72058af83ed0463b924b0ada17ee20b2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.077234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6e96a-b87b-4b5f-9754-e93eb8fcc0ec None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.078135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72058af83ed0463b924b0ada17ee20b2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.087128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.092326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.093007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.093289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.093545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.094321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:23.094641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.096258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.120666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 890/1807] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:23.124973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.135886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124bfaf-4085-4bc6-b293-26978f468426 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.143365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.144262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.144679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.145088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.146815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.149047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.206426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72058af83ed0463b924b0ada17ee20b2', 'request_id': 'req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.207104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 891/1808] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:23.209132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.225971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72058af83ed0463b924b0ada17ee20b2', 'request_id': 'req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.244488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72058af83ed0463b924b0ada17ee20b2', 'request_id': 'req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.246891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:23.289204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] The token cache is being invalidate because project 72058af83ed0463b924b0ada17ee20b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:23.289506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.292217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6b9c50bb-e545-4ab0-b8f7-dacf4cf07119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.297361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 918/1809] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:23 2022] DELETE /identity/v3/projects/72058af83ed0463b924b0ada17ee20b2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:23.303959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc7fec-f1cc-4aa8-9166-3383321262d1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.304245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc7fec-f1cc-4aa8-9166-3383321262d1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.304567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc7fec-f1cc-4aa8-9166-3383321262d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.347282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6dbc7fec-f1cc-4aa8-9166-3383321262d1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:23.349297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6dbc7fec-f1cc-4aa8-9166-3383321262d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.377173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0aba0-5c50-48e9-8462-83f1f96c4733 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.395667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.396455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.396591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.396831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.397473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.403571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.452427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 892/1810] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:23.453499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.456498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.484304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331223a1-303e-4576-b82a-e3ab0d0fedbc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.488715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 919/1811] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:23.493412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.494103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.494380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.494708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.495315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.497265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.497540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.497959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.500461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.520475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 893/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:23.528303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.546230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:23.547491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.548164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605cc66-4ba5-4d7a-81e3-66592a76fe9a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.563141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.564636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.564925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.565204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.571459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 920/1813] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:23.573929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.574330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.578823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.599499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bfdfd-6667-46c9-91eb-6c03338922f0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.604852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 894/1814] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:23.614771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.619121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.620796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.621180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.622287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.624624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:23.625406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.628005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.642758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c942ca-279f-43b5-acbc-b2ca19b85395 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.649276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 921/1815] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:23.651913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.653162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.653439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.653713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] PATH_INFO: `/v3/projects/0165d818f14b4923b22538f65df4aa5d/users/a356295e82144819b3da36d361b5c6b1/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.658828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.662630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:23.680175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f4e24-b8a0-4359-8f58-2620ee8b381b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.691433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0165d818f14b4923b22538f65df4aa5d, user_id=a356295e82144819b3da36d361b5c6b1, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.694119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.708186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:23.708996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.709583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.710038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] PATH_INFO: `/v3/projects/c623c591cae549339b3a0c1909752f3d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.711478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c623c591cae549339b3a0c1909752f3d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:23.714055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:23.718920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 895/1816] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:23 2022] PUT /identity/v3/projects/0165d818f14b4923b22538f65df4aa5d/users/a356295e82144819b3da36d361b5c6b1/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:23.728471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:23.806117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:23.835996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c623c591cae549339b3a0c1909752f3d', 'request_id': 'req-f94310c1-b5da-4785-8aa5-c71611528e75'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.854688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c623c591cae549339b3a0c1909752f3d', 'request_id': 'req-f94310c1-b5da-4785-8aa5-c71611528e75'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.866013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c623c591cae549339b3a0c1909752f3d', 'request_id': 'req-f94310c1-b5da-4785-8aa5-c71611528e75'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.876010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bebf94-37f5-4773-918c-8d2ae03f2cb2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:23.885493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] The token cache is being invalidate because project c623c591cae549339b3a0c1909752f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:23.886304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.888098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f94310c1-b5da-4785-8aa5-c71611528e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:23.891655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 922/1817] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:23 2022] DELETE /identity/v3/projects/c623c591cae549339b3a0c1909752f3d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:23.893561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ed11-2d14-42dd-8ec9-89d8d6867212 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:23.894469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ed11-2d14-42dd-8ec9-89d8d6867212 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:23.895494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527ed11-2d14-42dd-8ec9-89d8d6867212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:23.980687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2527ed11-2d14-42dd-8ec9-89d8d6867212 None None] MFA Rules not processed for user `a356295e82144819b3da36d361b5c6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:23.985495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2527ed11-2d14-42dd-8ec9-89d8d6867212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:24.119283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.120938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.121977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.122674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.124160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.130822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.179123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:24.201363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 923/1818] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:24.201363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.201363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.201363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.233573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 896/1819] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:23 2022] POST /identity/v3/projects => generated 401 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:24.233573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.235203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:24.270666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:24.272172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:24.428137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc554-a225-4eeb-b47e-6b8ea4a1e836 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.442016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.444536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.445610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.446675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.447435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.453878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 924/1820] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:24.456134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bf084-4d10-45c5-91d2-f2aa5ba4066b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.458019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.464202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:24.498772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51522d5-c026-4166-99d8-5049d3b6abec None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.514932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.517478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.517948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.521116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.532325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.538404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.540739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 897/1821] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/projects => generated 373 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:24.542304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.543758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.544830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.585076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:24.586778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:24.597543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 925/1822] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/users => generated 301 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:24.599159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.625268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e2d8-758f-40fd-a5b1-f1002aab2eaf admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.687623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 898/1823] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:24.688643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.702339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4037c7-25ff-484e-83a4-0d7f686a5e33 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.708497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.709238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.709617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.710732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.712091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.714136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.731960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:24.737683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 899/1824] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:24.737683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.761118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a064-3e49-45a9-855d-294a4a9f2d5d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.770873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.771774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.772589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.774213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.774405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.775388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.775759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.776178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.776858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.777515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:24.777994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.780802 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.783453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.798698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 900/1825] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:24.803519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.819174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fc805-754c-4834-bcb0-63e4adc33bb3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.830102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.831021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.831481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.831946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.832749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.832983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 926/1826] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:24.834955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.835408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.851449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 901/1827] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:24.853307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.853625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d9aad-70eb-40d9-b29b-562c37ef4acd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.863352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.864291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.865113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.865707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.867007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.867543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1cfaa-566b-4ed7-a127-51d2e4e009dc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.871837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.876069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.877121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.877554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.878012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.878818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.880734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.894785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 902/1828] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:24.896299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.914851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af4211-10ff-4f28-9faf-e5efe979515b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:24.922538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:24.923227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:24.923536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:24.923815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] PATH_INFO: `/v3/projects/69f342106fa745dea28aba7fd041e95c/users/02afe5d3222e455f8b768aab5e1c728f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:24.933123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 927/1829] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:24.934922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.951955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f342106fa745dea28aba7fd041e95c, user_id=02afe5d3222e455f8b768aab5e1c728f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:24.955796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:24.977786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 903/1830] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:24 2022] PUT /identity/v3/projects/69f342106fa745dea28aba7fd041e95c/users/02afe5d3222e455f8b768aab5e1c728f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:24.977786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:24.979798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:25.029401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418447-3ebb-4151-aec7-e9e087918244 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.095605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64005352-0e0a-434b-954a-fbc3e8c52664 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.102963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.103825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.104656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.104992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.106359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.108376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56722a06-a195-4869-8aee-94dfa49a63e9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.145780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 904/1831] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:24 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:25.146397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.146685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.146960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.173597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.174819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.175574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.175574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] PATH_INFO: `/v3/projects/44154c190dc84c999d7c31a0d536ee01/users/52bf70eba6ea48559988434527e2c1d1/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.189907 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] MFA Rules not processed for user `02afe5d3222e455f8b768aab5e1c728f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:25.191885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.201538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44154c190dc84c999d7c31a0d536ee01, user_id=52bf70eba6ea48559988434527e2c1d1, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.205752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.238425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 928/1832] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:24 2022] PUT /identity/v3/projects/44154c190dc84c999d7c31a0d536ee01/users/52bf70eba6ea48559988434527e2c1d1/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:25.240452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.242790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.281014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 905/1833] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:25.282732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.298772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:25.302900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a17ee-f7f4-40a3-b0b3-bea90256d1ac None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.343539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a417e8-9035-4cc2-82ed-4e833d610470 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.350602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.352767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.353280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.353751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] PATH_INFO: `/v3/projects/b50c5581c26446bab76b98797e021051` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.356311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50c5581c26446bab76b98797e021051)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.358464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.385615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.386379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.386893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.387333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.388070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.389815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.402034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 906/1834] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:25.405898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.416250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f43a-e425-4e8f-9b40-d93a17b86855 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.428208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50c5581c26446bab76b98797e021051', 'request_id': 'req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:25.435630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:25.437641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50c5581c26446bab76b98797e021051', 'request_id': 'req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:25.450241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50c5581c26446bab76b98797e021051', 'request_id': 'req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:25.464061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] The token cache is being invalidate because project b50c5581c26446bab76b98797e021051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:25.465289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:25.465715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:25.467700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 929/1835] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:25 2022] DELETE /identity/v3/projects/b50c5581c26446bab76b98797e021051 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:25.470567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.472433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.495104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.495997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.496628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.496976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.497693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.499520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.515025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 907/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:25.515950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.519978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.538153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba625b7f-f471-4267-98ab-359737333246 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.544492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.545308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.545593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.546070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.546563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.548315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69019e1-4b51-47fd-bb5d-2269b94bedb7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.552075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257b5ab-8a6f-4458-9dcc-b50af55f36f6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.557337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.558070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.558374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.558881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.559593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.560165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 908/1837] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:25.562048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.562336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.562828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.564795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.586371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:25.588140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.597784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 909/1838] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:25.598489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.610743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 930/1839] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/projects => generated 399 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:25.611862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.614731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.631641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4df619-68ba-4d84-bbc1-2f9a5e5f5c97 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.676373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:25.700643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908938b-6292-4917-8d54-942620bd08ca admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.705831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.706550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.706784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.707057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] PATH_INFO: `/v3/projects/68df61247b7a411cb3faf45a8667e8c8/users/92bf53e1f07644caa3bc664945bab7f5/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.713833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.714578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.715447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.715719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] PATH_INFO: `/v3/projects/44154c190dc84c999d7c31a0d536ee01/users/52bf70eba6ea48559988434527e2c1d1/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.718538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44154c190dc84c999d7c31a0d536ee01, user_id=52bf70eba6ea48559988434527e2c1d1, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.721488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba566cc-bcfd-4da6-ad7f-cfafb6134f26 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.727754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68df61247b7a411cb3faf45a8667e8c8, user_id=92bf53e1f07644caa3bc664945bab7f5, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.731274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.742760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 910/1840] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:25 2022] PUT /identity/v3/projects/44154c190dc84c999d7c31a0d536ee01/users/52bf70eba6ea48559988434527e2c1d1/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:25.744286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c6c9-f7a9-4d4d-97f7-b1bb022e025f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.744574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c6c9-f7a9-4d4d-97f7-b1bb022e025f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.744846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382c6c9-f7a9-4d4d-97f7-b1bb022e025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.756689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 931/1841] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:25 2022] PUT /identity/v3/projects/68df61247b7a411cb3faf45a8667e8c8/users/92bf53e1f07644caa3bc664945bab7f5/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:25.757512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:25.781394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5382c6c9-f7a9-4d4d-97f7-b1bb022e025f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:25.783556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5382c6c9-f7a9-4d4d-97f7-b1bb022e025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.850975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c462c-d8ed-4787-8282-9b04e48ada88 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:25.857935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:25.858556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.858837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.859108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.859878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:25.861969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddea5fd-7811-45e9-89e4-be2f4c0e52a6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:25.878681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 911/1842] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:25.878681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ced33f-c844-4902-981b-c2e18af250ee None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.878681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ced33f-c844-4902-981b-c2e18af250ee None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.878681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ced33f-c844-4902-981b-c2e18af250ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.899083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 932/1843] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/users => generated 306 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:25.900289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:25.900572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:25.900850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:25.925029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f2ced33f-c844-4902-981b-c2e18af250ee None None] MFA Rules not processed for user `309e981f1d944bc487e0d54b5a582ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:25.926391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2ced33f-c844-4902-981b-c2e18af250ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.940161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] MFA Rules not processed for user `52bf70eba6ea48559988434527e2c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:25.941378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:25.948660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:25.998518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 912/1844] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:25.999847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.000135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.000407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.017779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 933/1845] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:26.018798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.032865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434524ff-e356-4c8f-954a-a10f33d974d5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.035873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] MFA Rules not processed for user `92bf53e1f07644caa3bc664945bab7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:26.037394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:26.038429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.038982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.039336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.039672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.040400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:26.040772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.042498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.056155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 934/1846] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:26.057081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.072213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60be7e-21ac-4fb9-9b8e-d08c943b1ee3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.078740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.079547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.080131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.080254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.080937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.082667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.094267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 935/1847] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.098696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.100252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:26.128464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 913/1848] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:26.129132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:26.130116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.147921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d841e4-0909-4c77-8fa4-b66824acca4e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.154854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.155481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.156177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.156567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.157987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.161446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.175980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb0f7-783e-45b0-88c4-3a15b33a8165 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.180507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.181137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.181417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.181687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.192320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.193931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.194632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 914/1849] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:26 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:26.199326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.205340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 936/1850] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:26.207875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.222843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb756f-776b-4be6-8554-7fb6f27b721e service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.310821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.311462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.311745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.312076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.312723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.314803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.315081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7ed35-90fb-4f15-8fd8-e0176f83d891 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.322000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.323501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.323927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.324403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] PATH_INFO: `/v3/projects/b2406f7a61c7472dbb750bdbad5aa237/users/2c8f19050649444eb9a7a9110f8c0538/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.350674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2406f7a61c7472dbb750bdbad5aa237, user_id=2c8f19050649444eb9a7a9110f8c0538, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.352431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.353915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 937/1851] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:26 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:26.355908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.370552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 915/1852] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:26 2022] PUT /identity/v3/projects/b2406f7a61c7472dbb750bdbad5aa237/users/2c8f19050649444eb9a7a9110f8c0538/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:26.372325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.383948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:26.465986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd4676-24f9-4d70-baac-c30f51a8cdc3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.470549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957f07-4bb7-4af8-b551-c9858d1725b1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.471719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.472458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.472968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.473346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.474134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.476537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.480589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.481275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.481560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.481913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.482560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.484562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.504846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 916/1853] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:26 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:26.506051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.510832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 938/1854] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:26 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:26.512378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.519374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571b61-ea3f-4cab-a2be-35c1af83b041 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.525458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0a93d-08f9-4bd5-9ff3-2e5c623ea4f9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.525882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.526601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.526984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.527647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.528382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.530838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.531601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.532466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.532820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.533150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.534178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.535778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.546052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 917/1855] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.547045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.555407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 939/1856] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.556307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.563221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45649-6373-4a99-8185-80557e4ac31a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.568467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80eb9e2-da30-4598-bb6f-cc999c67eb86 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.568600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.569075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.569413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.569621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.570482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.572383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.576494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.577184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.577488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.577780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] PATH_INFO: `/v3/projects/b2406f7a61c7472dbb750bdbad5aa237/users/2c8f19050649444eb9a7a9110f8c0538/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.582747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2406f7a61c7472dbb750bdbad5aa237, user_id=2c8f19050649444eb9a7a9110f8c0538, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.585684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.589597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 918/1857] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.590755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.602141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55cd1a-4501-4ef1-a601-0300b3e5ec59 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.604408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 940/1858] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:26 2022] PUT /identity/v3/projects/b2406f7a61c7472dbb750bdbad5aa237/users/2c8f19050649444eb9a7a9110f8c0538/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:26.606248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.643727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:26.671500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.672404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.672776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.673152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] PATH_INFO: `/v3/projects/58f0673df7f94e118c719c7042ef8a6b/users/3255ddcda7364d2ea4fcf1dd705ae01c/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.694331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f0673df7f94e118c719c7042ef8a6b, user_id=3255ddcda7364d2ea4fcf1dd705ae01c, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.697697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a17ba-5aa2-42c2-8008-4d6408626b2f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.706555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8de7-951b-44cc-9481-2654432abe64 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.715386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 919/1859] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:26 2022] PUT /identity/v3/projects/58f0673df7f94e118c719c7042ef8a6b/users/3255ddcda7364d2ea4fcf1dd705ae01c/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:26.716646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.716927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.717200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.759881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] MFA Rules not processed for user `2c8f19050649444eb9a7a9110f8c0538`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:26.761146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:26.799081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.799715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.799993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.800266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.801500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.803343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.824528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 941/1860] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.825838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.843294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd3483-e859-4c7d-bb20-c89a9f061ab0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.843977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:26.853397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.854140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.854417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.854956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.856126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.857921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.869144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 942/1861] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:26.870030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.883766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c392-0903-4590-a300-f4a4fd3dde20 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.894389 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 920/1862] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:26.896080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:26.901164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.902478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.902984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.903476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] PATH_INFO: `/v3/users/a356295e82144819b3da36d361b5c6b1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.907200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a356295e82144819b3da36d361b5c6b1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.911957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:26.935082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7911cb7-5407-4dda-b06f-488ebc1e8cf5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:26.943628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:26.944254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:26.944686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:26.945104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] PATH_INFO: `/v3/projects/545f206f0a0343c9bb29942819524e7d/users/c2608b9885c14afda0841ece6fbec266/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:26.989759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545f206f0a0343c9bb29942819524e7d, user_id=c2608b9885c14afda0841ece6fbec266, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:26.998119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.033848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a356295e82144819b3da36d361b5c6b1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:27.036906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 921/1863] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:26 2022] PUT /identity/v3/projects/545f206f0a0343c9bb29942819524e7d/users/c2608b9885c14afda0841ece6fbec266/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:27.037761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.046386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a356295e82144819b3da36d361b5c6b1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:27.047079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a356295e82144819b3da36d361b5c6b1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:27.047742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a356295e82144819b3da36d361b5c6b1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:27.056033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a356295e82144819b3da36d361b5c6b1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:27.076157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 943/1864] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:26 2022] DELETE /identity/v3/users/a356295e82144819b3da36d361b5c6b1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:27.080116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.082235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.117641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:27.123026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bc7a7-cca5-45f9-93ce-c129a32d8726 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.174599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e9aaf-f4cd-4771-b827-e3f43c318ddc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.180579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.182173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.182463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.182740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.185837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.197472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.198512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.198897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.199301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] PATH_INFO: `/v3/projects/58f0673df7f94e118c719c7042ef8a6b/users/3255ddcda7364d2ea4fcf1dd705ae01c/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.201652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f0673df7f94e118c719c7042ef8a6b, user_id=3255ddcda7364d2ea4fcf1dd705ae01c, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.204669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37a3ba-1b55-41f5-89d7-780a2ba1d281 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.219769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 922/1865] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:27 2022] PUT /identity/v3/projects/58f0673df7f94e118c719c7042ef8a6b/users/3255ddcda7364d2ea4fcf1dd705ae01c/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:27.221078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1fccf-f3f5-4056-8547-1433abf32207 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.221354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1fccf-f3f5-4056-8547-1433abf32207 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.221628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1fccf-f3f5-4056-8547-1433abf32207 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.223035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 923/1866] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:27.224053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31498-3b1c-4c01-91ca-4c21f72e19e0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.224604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31498-3b1c-4c01-91ca-4c21f72e19e0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.224889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31498-3b1c-4c01-91ca-4c21f72e19e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.263814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3cc31498-3b1c-4c01-91ca-4c21f72e19e0 None None] MFA Rules not processed for user `c2608b9885c14afda0841ece6fbec266`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:27.265052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3cc31498-3b1c-4c01-91ca-4c21f72e19e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.281653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.283644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ace0-d47e-4db0-b2f8-5212f90ce8d6 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.299209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 944/1867] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:27 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:27.300754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb143-8fa8-463b-b284-fc79017ccb15 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.301351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb143-8fa8-463b-b284-fc79017ccb15 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.301636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb143-8fa8-463b-b284-fc79017ccb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.328204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4d2eb143-8fa8-463b-b284-fc79017ccb15 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:27.329212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:27.330062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4d2eb143-8fa8-463b-b284-fc79017ccb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.361642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 924/1868] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:27.363206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.363498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.363774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.385603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:27.387412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.437684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 945/1869] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:27.439243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.439535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.439814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.473784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] MFA Rules not processed for user `3255ddcda7364d2ea4fcf1dd705ae01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:27.475660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.484654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 925/1870] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:27.485515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.487105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.502113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397964c-300f-4c9a-a904-b20aa4d979b4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.507713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.508479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.508854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.509233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.510080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.512654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.549029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 926/1871] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:27.549029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.576067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:27.594058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 946/1872] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:27.595506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.601983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.626170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ef3a4-4519-41d0-96cd-cc09ce9807a6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.631998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.632835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.633260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.633691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.635875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.675535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a01fea-8665-45f6-813e-d31510d256ef None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.724580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.726432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.737248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 927/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:27 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:27.738192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.739834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.755902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.757078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.757674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.758140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.770075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:27.855902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a1fb-78b8-4992-8685-fae1660adcdd service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.862569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.863265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.863606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.863733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.864267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.865556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.875999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.877132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 947/1874] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:27.877823 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.878381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.899643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc673b-cc70-4d3e-b61e-1e8cfda996a6 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.900146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 928/1875] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:27.902040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.904155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.909825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.910873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.911212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.911489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.912140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.915855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:27.919463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04c68-cf3b-453b-93ad-fa96cb2fa9a3 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.924401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.925023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.925306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.925719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.955148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 948/1876] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:27 2022] POST /identity/v3/users => generated 297 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:27.955148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:27.958938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:27.975445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf5116-7965-4a4e-923e-65541185d6f9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:27.981282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:27.981983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:27.982263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:27.982697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] PATH_INFO: `/v3/users/6d056384262444cf903eb876987f3b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:27.983959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d056384262444cf903eb876987f3b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:27.986034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.022029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.026605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.046637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:28.053022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 929/1877] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:28.055992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d056384262444cf903eb876987f3b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.057247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.073413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d056384262444cf903eb876987f3b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.073784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d056384262444cf903eb876987f3b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.075893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d056384262444cf903eb876987f3b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.112369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3f0d9-cbde-452d-bcbb-d38f835affef service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.115514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d056384262444cf903eb876987f3b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.126862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.127551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.127945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.128332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.129501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.130956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.147223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 930/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:28.148380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.166654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.175211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 949/1879] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:27 2022] DELETE /identity/v3/users/6d056384262444cf903eb876987f3b18 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:28.176149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.178877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.200167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b6f04-bb35-4140-aa62-060ce84798b4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.355452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.355998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.356426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.357131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] PATH_INFO: `/v3/projects/be463be8e72c4521b0668422cb52e8cb/users/983361925d1a42feb3ed90cb847fc1a3/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.361114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a4636-f481-4dd9-b9a0-9df6d968b04c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.367924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.368670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.369048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.369398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] PATH_INFO: `/v3/users/92bf53e1f07644caa3bc664945bab7f5` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.370767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92bf53e1f07644caa3bc664945bab7f5)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.372824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.411196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be463be8e72c4521b0668422cb52e8cb, user_id=983361925d1a42feb3ed90cb847fc1a3, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.414312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.442838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 931/1880] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:28 2022] PUT /identity/v3/projects/be463be8e72c4521b0668422cb52e8cb/users/983361925d1a42feb3ed90cb847fc1a3/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:28.442838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.445382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.451821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92bf53e1f07644caa3bc664945bab7f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.463231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92bf53e1f07644caa3bc664945bab7f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.464309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92bf53e1f07644caa3bc664945bab7f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.466697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92bf53e1f07644caa3bc664945bab7f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.476728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92bf53e1f07644caa3bc664945bab7f5'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.497166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:28.513599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 950/1881] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:28 2022] DELETE /identity/v3/users/92bf53e1f07644caa3bc664945bab7f5 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:28.514725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.517572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.576463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:28.610356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385beb9-cfed-46a0-8ed3-dbf2a1a0c99d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.614515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40590f4-8d4e-4806-9492-2ed394140c96 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.622685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.624569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.624963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.625830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.625952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.626290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.626401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] PATH_INFO: `/v3/projects/0165d818f14b4923b22538f65df4aa5d` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.626570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.627826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.628853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0165d818f14b4923b22538f65df4aa5d)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.631261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.634243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e96eb-8140-469b-b595-d0420d15a467 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.655005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 951/1882] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:28.656358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.656651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.656926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.706731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:28.708282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.718498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 952/1883] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:28.720531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.735127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45fbb5-fb04-493a-9d4d-5bcbb7664026 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.754942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0165d818f14b4923b22538f65df4aa5d', 'request_id': 'req-3fa273bb-d85a-446d-b72b-e7f54dd4e786'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.778758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0165d818f14b4923b22538f65df4aa5d', 'request_id': 'req-3fa273bb-d85a-446d-b72b-e7f54dd4e786'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.787570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0165d818f14b4923b22538f65df4aa5d', 'request_id': 'req-3fa273bb-d85a-446d-b72b-e7f54dd4e786'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.808256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] The token cache is being invalidate because project 0165d818f14b4923b22538f65df4aa5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:28.809318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.810209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:28.812939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 932/1884] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:28 2022] DELETE /identity/v3/projects/0165d818f14b4923b22538f65df4aa5d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:28.813808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.816139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.845614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.846976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.847448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.848037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] PATH_INFO: `/v3/projects/be463be8e72c4521b0668422cb52e8cb/users/983361925d1a42feb3ed90cb847fc1a3/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.851153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be463be8e72c4521b0668422cb52e8cb, user_id=983361925d1a42feb3ed90cb847fc1a3, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.856069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:28.885899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 953/1885] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:28 2022] PUT /identity/v3/projects/be463be8e72c4521b0668422cb52e8cb/users/983361925d1a42feb3ed90cb847fc1a3/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:28.887042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:28.891208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.911385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa273bb-d85a-446d-b72b-e7f54dd4e786 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.962883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9c293-c37f-4ff5-88e8-0532415cd011 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:28.967137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.967915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.968483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.969239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.971838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:28.972585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:28.992094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:28.992755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:28.993033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:28.993303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] PATH_INFO: `/v3/projects/28b867f93c7741519684967090242468` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:28.994977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b867f93c7741519684967090242468)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:28.998112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:29.056324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:29.057947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:29.059382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b867f93c7741519684967090242468', 'request_id': 'req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.067736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b867f93c7741519684967090242468', 'request_id': 'req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.069835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 954/1886] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:29.070761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:29.072349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:29.076860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b867f93c7741519684967090242468', 'request_id': 'req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.087078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015558-fa3a-4fab-a0ef-78c4a3cafce0 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:29.091311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] The token cache is being invalidate because project 28b867f93c7741519684967090242468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:29.091833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.092552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:29.093908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a63f42ff-dd03-4d5c-895b-25f6cbfd4ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.100067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 933/1887] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:28 2022] DELETE /identity/v3/projects/28b867f93c7741519684967090242468 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:29.102087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:29.102469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:29.102743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:29.147020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] MFA Rules not processed for user `983361925d1a42feb3ed90cb847fc1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:29.148812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:29.166290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:29.167072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:29.168579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:29.168858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] PATH_INFO: `/v3/users/52bf70eba6ea48559988434527e2c1d1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:29.170810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bf70eba6ea48559988434527e2c1d1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:29.174541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:29.260653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 934/1888] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:29.261444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:29.264398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bf70eba6ea48559988434527e2c1d1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.269345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:29.278519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bf70eba6ea48559988434527e2c1d1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.279672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bf70eba6ea48559988434527e2c1d1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.280535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bf70eba6ea48559988434527e2c1d1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.283504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e14b3-2ca9-47bb-9f2b-6b5f141a923a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:29.288559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:29.289205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:29.289480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:29.289775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] PATH_INFO: `/v3/users/309e981f1d944bc487e0d54b5a582ce6` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:29.291063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309e981f1d944bc487e0d54b5a582ce6)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:29.291309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bf70eba6ea48559988434527e2c1d1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.293657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:29.320487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 955/1889] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:29 2022] DELETE /identity/v3/users/52bf70eba6ea48559988434527e2c1d1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:29.320487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:29.323152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:29.365043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309e981f1d944bc487e0d54b5a582ce6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.380046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309e981f1d944bc487e0d54b5a582ce6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.382312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309e981f1d944bc487e0d54b5a582ce6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.383065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309e981f1d944bc487e0d54b5a582ce6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.392443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309e981f1d944bc487e0d54b5a582ce6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:29.393938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:29.447268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 935/1890] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:29 2022] DELETE /identity/v3/users/309e981f1d944bc487e0d54b5a582ce6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:29.448591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a6342-6043-4ad2-b0cf-3ffa817f9738 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:29.509898 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:29.521647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:29.522571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:29.523129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:29.523611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:29.526680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:29.638264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:29.642607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:29.669406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 956/1891] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:30.025496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.030579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.156986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.158694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.180315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea520e-3893-4d66-a0dc-7f2e593e1539 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.187240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.188228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.188651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.188933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] PATH_INFO: `/v3/projects/778e78e5e3364655931f1bc13d69a7cc` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.190378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] RBAC: Authorizing `identity:delete_project(project_id=778e78e5e3364655931f1bc13d69a7cc)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.192632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.279711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778e78e5e3364655931f1bc13d69a7cc', 'request_id': 'req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.287661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778e78e5e3364655931f1bc13d69a7cc', 'request_id': 'req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.303724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778e78e5e3364655931f1bc13d69a7cc', 'request_id': 'req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.326543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] The token cache is being invalidate because project 778e78e5e3364655931f1bc13d69a7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:30.326813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.327496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7c1c4dd7-4fa0-484f-b9a0-af5868d9291b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.338716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 936/1892] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:30 2022] DELETE /identity/v3/projects/778e78e5e3364655931f1bc13d69a7cc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:30.340513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.340701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.340936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.364265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:30.365645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.366325 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab65ec3-96a5-49b6-b718-f3fa707272c5 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.440411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 937/1893] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:30.442956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.444610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.475531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be3ef-71a5-4c43-8ede-c323421ffbb4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.481074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.481631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.482059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.482342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] PATH_INFO: `/v3/projects/44154c190dc84c999d7c31a0d536ee01` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.483501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=44154c190dc84c999d7c31a0d536ee01)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.485455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.486578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.487304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.487737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.488006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] PATH_INFO: `/v3/projects/68df61247b7a411cb3faf45a8667e8c8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.489142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=68df61247b7a411cb3faf45a8667e8c8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.491125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.492505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:30.535310 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44154c190dc84c999d7c31a0d536ee01', 'request_id': 'req-14bbfadb-f8c1-402a-9341-78a3529a9ccd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.537244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:30.542320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68df61247b7a411cb3faf45a8667e8c8', 'request_id': 'req-fd37afbb-c78b-4e09-bfeb-b717520932cb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.543897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44154c190dc84c999d7c31a0d536ee01', 'request_id': 'req-14bbfadb-f8c1-402a-9341-78a3529a9ccd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.550582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68df61247b7a411cb3faf45a8667e8c8', 'request_id': 'req-fd37afbb-c78b-4e09-bfeb-b717520932cb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.553104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44154c190dc84c999d7c31a0d536ee01', 'request_id': 'req-14bbfadb-f8c1-402a-9341-78a3529a9ccd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.559689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68df61247b7a411cb3faf45a8667e8c8', 'request_id': 'req-fd37afbb-c78b-4e09-bfeb-b717520932cb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.565599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] The token cache is being invalidate because project 44154c190dc84c999d7c31a0d536ee01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:30.566026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.566666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.568364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 938/1894] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:30 2022] DELETE /identity/v3/projects/44154c190dc84c999d7c31a0d536ee01 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:30.569044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.570757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.570882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] The token cache is being invalidate because project 68df61247b7a411cb3faf45a8667e8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:30.570882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.572170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fd37afbb-c78b-4e09-bfeb-b717520932cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.573498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 957/1895] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:30 2022] DELETE /identity/v3/projects/68df61247b7a411cb3faf45a8667e8c8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:30.574830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b69c12-1dc6-47de-8481-92c7e5339db9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.575323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b69c12-1dc6-47de-8481-92c7e5339db9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.575598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b69c12-1dc6-47de-8481-92c7e5339db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.595075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-49b69c12-1dc6-47de-8481-92c7e5339db9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:30.596269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-49b69c12-1dc6-47de-8481-92c7e5339db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.639171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbfadb-f8c1-402a-9341-78a3529a9ccd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.657488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 958/1896] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:30.658773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.659050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.659314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.678603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:30.679788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.688303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 959/1897] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:30.689001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.690227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.698767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983361925d1a42feb3ed90cb847fc1a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be463be8e72c4521b0668422cb52e8cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.699412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.699693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.699962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.700605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:30.700906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.701822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b142569-252c-4d05-b84e-7634fe16674c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.702554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.706400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.707011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.707284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.707550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] PATH_INFO: `/v3/users/3255ddcda7364d2ea4fcf1dd705ae01c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.708666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3255ddcda7364d2ea4fcf1dd705ae01c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.710596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.715582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 939/1898] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:30.716246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.726352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bff96-7ce6-4b0f-b6e1-c3bc4e1f1bfc tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.731626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.732210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.732486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.732755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.733383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:30.733681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.735383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0e5cd-dc23-40fb-87c0-95a48894dab4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.749105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 940/1899] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:30.750194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.750585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.750924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.763063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3255ddcda7364d2ea4fcf1dd705ae01c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.771157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3255ddcda7364d2ea4fcf1dd705ae01c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.771792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3255ddcda7364d2ea4fcf1dd705ae01c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.772429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3255ddcda7364d2ea4fcf1dd705ae01c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.773248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:30.774496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.780227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3255ddcda7364d2ea4fcf1dd705ae01c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:30.783202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 941/1900] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:30.787207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.788555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.800005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 960/1901] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:30 2022] DELETE /identity/v3/users/3255ddcda7364d2ea4fcf1dd705ae01c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:30.801939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c83b6-f3b1-49a5-bcae-02e3661d6ae0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.806037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.807507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:30.813767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:30.861373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:30.893058 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983361925d1a42feb3ed90cb847fc1a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be463be8e72c4521b0668422cb52e8cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.893710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.894017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.894293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.899340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.899340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.916065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088fed9-9d3d-4d85-8c88-5563cf0d7b06 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:30.922355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:30.922752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:30.923032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:30.923301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:30.924077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:30.926084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:30.973327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 942/1902] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/projects => generated 355 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:30.977336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 961/1903] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] POST /identity/v3/projects => generated 357 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:30.978855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:30.979370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.166278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c235cdc-b4f4-4b11-a83a-889dcb2090c2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.174899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d140-8649-4a4e-9ea1-4792b1ebd44e tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.185670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983361925d1a42feb3ed90cb847fc1a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be463be8e72c4521b0668422cb52e8cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.187139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.188668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.189173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.190027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:31.192334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.194364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.218256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 962/1904] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:31.218256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.324730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.325648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.326813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.327747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.329210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:31.329705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.331826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.333510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24511bda-27df-44c3-9022-73c6c969679a tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.339253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.340822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.340822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.341079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.342337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.344389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.354479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 943/1905] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:31.356404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.368304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:31.384074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2c627-efed-40d7-911e-2c75eaecf1c9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.392204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 963/1906] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:31 2022] POST /identity/v3/users => generated 279 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:31.392204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.401685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.405911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.407435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee4b90-2e82-41f7-8b5f-c1cc02b4dde8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.409562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.409976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] PATH_INFO: `/v3/projects/58f0673df7f94e118c719c7042ef8a6b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.411375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f0673df7f94e118c719c7042ef8a6b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.412726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '983361925d1a42feb3ed90cb847fc1a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be463be8e72c4521b0668422cb52e8cb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.413458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.413802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.414161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.414580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] PATH_INFO: `/v3/projects/7330fc064d504e9ba02d15f0022b28b2` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.417091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7330fc064d504e9ba02d15f0022b28b2)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.419240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.464353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:31.505844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f0673df7f94e118c719c7042ef8a6b', 'request_id': 'req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.517991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f0673df7f94e118c719c7042ef8a6b', 'request_id': 'req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.527095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:31.542422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7330fc064d504e9ba02d15f0022b28b2', 'request_id': 'req-90be0b8a-bf12-402b-86fb-f9e6df524430'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.543527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f0673df7f94e118c719c7042ef8a6b', 'request_id': 'req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.558832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7330fc064d504e9ba02d15f0022b28b2', 'request_id': 'req-90be0b8a-bf12-402b-86fb-f9e6df524430'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.566388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] The token cache is being invalidate because project 58f0673df7f94e118c719c7042ef8a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:31.566722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.568787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7330fc064d504e9ba02d15f0022b28b2', 'request_id': 'req-90be0b8a-bf12-402b-86fb-f9e6df524430'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.571553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.574851 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 944/1907] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:31 2022] DELETE /identity/v3/projects/58f0673df7f94e118c719c7042ef8a6b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:31.580675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.584070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:31.584849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] The token cache is being invalidate because project 7330fc064d504e9ba02d15f0022b28b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:31.596206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.597773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:31.600463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 964/1908] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:31 2022] DELETE /identity/v3/projects/7330fc064d504e9ba02d15f0022b28b2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:31.601090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.603560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:31.756526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575d51-2fcd-40e3-93bf-2be6bc265ba4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.804806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0b8a-bf12-402b-86fb-f9e6df524430 tempest-VolumeQuotaClassesTest-2077292455 tempest-VolumeQuotaClassesTest-2077292455-project-admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.816722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.818880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.819421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.819920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.821315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.824908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329d807-b979-4615-ae7b-556750f4c422 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.847806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 965/1909] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:31.850366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.851280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.851769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.900689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:31.903877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:31.906938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.908297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.909806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.910148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.910732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.915881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac39b5f-bc7c-4bc5-b7aa-6ab0690d2350 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.928060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 966/1910] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:31.928994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.943919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7acac-996f-4057-ae92-a28fd9587d1d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.950213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:31.951334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.951615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.951905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.952565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:31.954239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:31.962510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 945/1911] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:31 2022] POST /identity/v3/projects => generated 385 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:31.963774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:31.964056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:31.964326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:31.973855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 967/1912] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:31.973855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:31.974418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:31.987267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:31.988224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e2c2-08a4-457c-9c2c-398c74277a71 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:31.988465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.044434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.050175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.050855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.051136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.051464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] PATH_INFO: `/v3/users/c2608b9885c14afda0841ece6fbec266` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.052667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2608b9885c14afda0841ece6fbec266)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.054661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.057549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 946/1913] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:32.058465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.059732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.071486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e226af-58d5-4d21-9f74-e349007f1970 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.076801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.077400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.077680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.077980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.078609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.080535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.101543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 947/1914] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.102387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.109576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2608b9885c14afda0841ece6fbec266'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:32.112134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.112911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e34ab-e13a-492f-ac3f-51f82fb78654 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.117337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2608b9885c14afda0841ece6fbec266'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:32.118183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2608b9885c14afda0841ece6fbec266'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:32.118922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.119374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2608b9885c14afda0841ece6fbec266'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:32.119749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.120115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.120452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] PATH_INFO: `/v3/projects/ff2ad7457a0e4776a12533624801bd21/users/ab716cb3074b486aa32ecbe078c0f80a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.127147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2608b9885c14afda0841ece6fbec266'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:32.135636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2ad7457a0e4776a12533624801bd21, user_id=ab716cb3074b486aa32ecbe078c0f80a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.138865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.145518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 968/1915] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:31 2022] DELETE /identity/v3/users/c2608b9885c14afda0841ece6fbec266 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:32.146487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.147918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.156033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 948/1916] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:32 2022] PUT /identity/v3/projects/ff2ad7457a0e4776a12533624801bd21/users/ab716cb3074b486aa32ecbe078c0f80a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:32.156723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.166853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.213659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49f376-64f7-4e0b-ade2-71d758c0a3ca admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.232737 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375716e-7834-40c0-b2ba-1d9c91fbfbe2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.237388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.238035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.238415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.238597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.239379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.241075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11adcb-026b-40fb-ac3d-d012dc70a1d5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.251713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 949/1917] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:32.252916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303ccc-14ac-4e4f-a639-18dd02b7b1bf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.253207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303ccc-14ac-4e4f-a639-18dd02b7b1bf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.253482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303ccc-14ac-4e4f-a639-18dd02b7b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.272291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.273123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.273273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ff303ccc-14ac-4e4f-a639-18dd02b7b1bf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:32.273459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.273756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.274384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ff303ccc-14ac-4e4f-a639-18dd02b7b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.274490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:32.274734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.276328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.282071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 950/1918] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:32.283235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.283514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.283782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.294704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 969/1919] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:32.294704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.303691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f163a2-91d3-4cdb-8830-63cd9c0d3ed0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.308928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.309533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.309873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.310152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.310815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.312400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.325621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 970/1920] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:32.326489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.328202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.330647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] MFA Rules not processed for user `ab716cb3074b486aa32ecbe078c0f80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:32.331997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.392884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d1fac-8d50-4f42-ae6e-849533836c0b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.396660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.397272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.397551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.397858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.407093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 951/1921] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:32.407093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.408376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.410574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.416443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.419207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d574-21aa-409e-8ca6-b7a505bfb125 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.422224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 971/1922] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:32.423026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.424784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.425568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.425879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.426155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.426823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.429643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.434512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fe410-79dc-4a2f-b155-76aa821626b4 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.440494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.441284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.441690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.442001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] PATH_INFO: `/v3/projects/3c7a58e3c6ab48bdb36fec632121b622/users/ed56f9ca03814844a13a12cfa9d8674e/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.463543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 952/1923] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.463902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7a58e3c6ab48bdb36fec632121b622, user_id=ed56f9ca03814844a13a12cfa9d8674e, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.463998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.464082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.467549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.486115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.486270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 972/1924] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:32 2022] PUT /identity/v3/projects/3c7a58e3c6ab48bdb36fec632121b622/users/ed56f9ca03814844a13a12cfa9d8674e/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:32.498555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.548689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97fc0a-a026-4c4a-8d34-41fb7ce42a84 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.577200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b172be-54e4-4569-a7ef-c892d40506cc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.582196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.582881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.583155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.583422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.584060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.586117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa9c0e-827f-4ef7-8326-f5dacf59ab0a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.613214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.614252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.617566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 973/1925] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.618048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.618048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.618048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.618515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.618515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.618515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.619082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.650718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] MFA Rules not processed for user `ed56f9ca03814844a13a12cfa9d8674e`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:32.651946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:32.653218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 953/1926] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/projects => generated 355 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.654253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.731142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f78f7-0273-4314-9736-a2dcca5276ab admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.734811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 974/1927] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:32.735735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.740328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.741275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.742010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.742528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.743957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.746710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.750578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eda6f7-6716-440b-888b-da7178a4c955 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.761332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.762222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.762605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.762984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.765024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 954/1928] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:32.766393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.769022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.771570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.780920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2bc83-2841-4562-956f-70151142800a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.789332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.790300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.790725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.791241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.791938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.795825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.806507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 975/1929] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/users => generated 284 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.807589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.821780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da224ed-84fe-481a-9172-6130b7bced73 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.835706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.836316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.836595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.836860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.837497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:32.839531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:32.846679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 955/1930] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:32.851558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.862512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 976/1931] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:32.863835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:32.879125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d75462-1f13-44fe-9cb2-84bd6ec83301 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.916420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.951163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:32.987576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31fbe6-54d6-4b86-bb6e-119efab9171f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:32.995044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:32.995708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:32.996728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:32.997083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:32.997856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.006555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.007593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.008049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.008506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.009780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.012637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.016009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.042198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 977/1933] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:32 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:33.045921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.046571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 956/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:33.047820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.050562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.063749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d3a7-8c19-4558-8f33-e44270e3a300 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.065464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a008a32-cfba-4f1b-acc6-3b53cc11fc79 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.069752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.070490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.070622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.070969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.071563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] PATH_INFO: `/v3/projects/c184cbb654a44118b2b77bf5609cb459/users/91dca5fb81b24cac8ce9e18dc5d89f72/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.071675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.071952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.072286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] PATH_INFO: `/v3/projects/545f206f0a0343c9bb29942819524e7d` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.073480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=545f206f0a0343c9bb29942819524e7d)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.076591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.091000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c184cbb654a44118b2b77bf5609cb459, user_id=91dca5fb81b24cac8ce9e18dc5d89f72, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.093959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.112052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 978/1934] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:33 2022] PUT /identity/v3/projects/c184cbb654a44118b2b77bf5609cb459/users/91dca5fb81b24cac8ce9e18dc5d89f72/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.113011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.115116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.134665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545f206f0a0343c9bb29942819524e7d', 'request_id': 'req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.142145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545f206f0a0343c9bb29942819524e7d', 'request_id': 'req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.151349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545f206f0a0343c9bb29942819524e7d', 'request_id': 'req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.163816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] The token cache is being invalidate because project 545f206f0a0343c9bb29942819524e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:33.164107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.165212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.167315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 957/1935] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:33 2022] DELETE /identity/v3/projects/545f206f0a0343c9bb29942819524e7d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.168481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.171493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.188708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02901e2-8183-4d51-a051-1d5ed579e1d4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.235654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627ea6f-45f6-4832-ab7e-68abc1f0cd2c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.240370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.241039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.241312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.241591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] PATH_INFO: `/v3/projects/57bef94217e044039a21e2e41f5081de/users/30c72ca127dc4b30b1b46402662c9b3d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.247970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.248543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.248818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.249085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] PATH_INFO: `/v3/users/02afe5d3222e455f8b768aab5e1c728f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.250280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] RBAC: Authorizing `identity:delete_user(user_id=02afe5d3222e455f8b768aab5e1c728f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.252662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.257102 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bef94217e044039a21e2e41f5081de, user_id=30c72ca127dc4b30b1b46402662c9b3d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.260147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.275101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 958/1936] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:33 2022] PUT /identity/v3/projects/57bef94217e044039a21e2e41f5081de/users/30c72ca127dc4b30b1b46402662c9b3d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.275979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.277779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.295888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:33.302076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02afe5d3222e455f8b768aab5e1c728f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.305309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:33.313112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02afe5d3222e455f8b768aab5e1c728f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.313994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02afe5d3222e455f8b768aab5e1c728f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.314749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02afe5d3222e455f8b768aab5e1c728f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.322142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-11376870-8ac9-4efd-a2f1-a1963e86d2db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02afe5d3222e455f8b768aab5e1c728f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.340225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 979/1937] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:33 2022] DELETE /identity/v3/users/02afe5d3222e455f8b768aab5e1c728f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.341496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.341932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.342267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.348817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155109-b6c3-4db8-b365-1c7eea0bf2e7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.370258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] MFA Rules not processed for user `91dca5fb81b24cac8ce9e18dc5d89f72`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:33.372058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.404419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.405019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.405298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.405563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.406260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.407838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.417474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 959/1938] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:33.418818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.422543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.434765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64bff5-2d25-4ae3-9a70-957efb9e2aef admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.437484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 980/1939] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:33.438303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.439619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.442761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.443401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.443689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.443962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] PATH_INFO: `/v3/users/983361925d1a42feb3ed90cb847fc1a3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.445149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983361925d1a42feb3ed90cb847fc1a3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.447368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.454004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcb73c-6b87-4ef3-808f-098927ae26e5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.458784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.459489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.459872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.460250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.461005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.462633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fdbaa-d264-48db-973e-817f610c014a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.472589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 981/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:33.473818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.474127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.474441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.494229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:33.495463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.497760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983361925d1a42feb3ed90cb847fc1a3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.504105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 982/1941] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:33.504895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.509827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983361925d1a42feb3ed90cb847fc1a3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.510497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983361925d1a42feb3ed90cb847fc1a3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.511125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983361925d1a42feb3ed90cb847fc1a3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.515165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.519947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983361925d1a42feb3ed90cb847fc1a3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:33.527307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78552226-3ffb-43c7-9345-082bd33db7ad None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.532404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.533069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.533541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.533947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] PATH_INFO: `/v3/projects/4886d00185ba46d4ac095e48da1fe056/users/03359f27225c47d18ac5b0c776ecea68/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.545274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 960/1942] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:33 2022] DELETE /identity/v3/users/983361925d1a42feb3ed90cb847fc1a3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.546718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.554280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4886d00185ba46d4ac095e48da1fe056, user_id=03359f27225c47d18ac5b0c776ecea68, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.557048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.564473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.584461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 983/1943] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:33 2022] PUT /identity/v3/projects/4886d00185ba46d4ac095e48da1fe056/users/03359f27225c47d18ac5b0c776ecea68/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.586270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.595560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.638215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:33.689932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb309ce-5a78-42a3-a3e3-c58eb2a52557 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.707554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e781ca-8585-4501-853a-d775d34045cb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.713217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.714096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.714449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.715335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] PATH_INFO: `/v3/projects/57bef94217e044039a21e2e41f5081de/users/30c72ca127dc4b30b1b46402662c9b3d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.717778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bef94217e044039a21e2e41f5081de, user_id=30c72ca127dc4b30b1b46402662c9b3d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.720788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.729863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:33.739557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 984/1944] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:33 2022] PUT /identity/v3/projects/57bef94217e044039a21e2e41f5081de/users/30c72ca127dc4b30b1b46402662c9b3d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:33.742284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.744153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.795179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.796122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.796122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.796319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.796967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.798918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.820772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603b7ba-3ab4-404f-8fd3-8d59a4639994 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.826199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.827148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.827648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.828381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.832465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.857705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 961/1945] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:33 2022] POST /identity/v3/projects => generated 389 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:33.857705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:33.864256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:33.953704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.955326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe58b-7811-4c97-83b5-1209fda18626 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.959425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816f19-bb28-4999-83ff-37fdca477ef8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:33.968140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:33.968593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.969049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.969326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.970021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:33.970238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 985/1946] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:33.971816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.972107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.972225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78a567-b718-48b1-a318-5ea04d54f4fe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:33.973086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.985370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 962/1947] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:33.992175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c08c5f-b31d-475b-991f-473b0976753f None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:33.992472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c08c5f-b31d-475b-991f-473b0976753f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:33.992754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c08c5f-b31d-475b-991f-473b0976753f None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:33.995835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 963/1948] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 07:59:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:59:33.998145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c08c5f-b31d-475b-991f-473b0976753f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.013578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] MFA Rules not processed for user `30c72ca127dc4b30b1b46402662c9b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:34.014829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.015600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c08c5f-b31d-475b-991f-473b0976753f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.025371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.025962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.026241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.026510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.027643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.030182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.065335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 964/1949] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:33 2022] POST /identity/v3/users => generated 301 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:34.070965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.088321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9de84-12bc-4142-9cae-d8eef36665a7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.102347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.103030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.103604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.103877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] PATH_INFO: `/v3/projects/4886d00185ba46d4ac095e48da1fe056/users/03359f27225c47d18ac5b0c776ecea68/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.108182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4886d00185ba46d4ac095e48da1fe056, user_id=03359f27225c47d18ac5b0c776ecea68, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.111869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701200b-0e1e-45b2-af78-7b4833e25e93 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.136902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 965/1950] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:34 2022] PUT /identity/v3/projects/4886d00185ba46d4ac095e48da1fe056/users/03359f27225c47d18ac5b0c776ecea68/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:34.138403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.138690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.138963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.162555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:34.164049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.171089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 986/1951] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:34.171089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.183093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d7032-2de2-4003-9136-592e7c03a83a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.203026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:34.248415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:34.257869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 966/1952] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 07:59:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:34.259179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.260713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.317360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.318415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.318954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.319548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.320590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.324592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e800d0-9018-4815-bfd3-5f3597f99ce5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.340520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 987/1953] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:34.342195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.342817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.343197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.370405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806bd9d-e014-4f12-9ca6-625bc9d13927 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.377867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.378799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.379081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.379350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] PATH_INFO: `/v3/projects/be463be8e72c4521b0668422cb52e8cb` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.380623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be463be8e72c4521b0668422cb52e8cb)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.382653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.406433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] MFA Rules not processed for user `03359f27225c47d18ac5b0c776ecea68`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:34.407571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.468732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be463be8e72c4521b0668422cb52e8cb', 'request_id': 'req-567e037b-504b-4834-a4de-3c7fa0674b23'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:34.476747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be463be8e72c4521b0668422cb52e8cb', 'request_id': 'req-567e037b-504b-4834-a4de-3c7fa0674b23'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:34.491090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be463be8e72c4521b0668422cb52e8cb', 'request_id': 'req-567e037b-504b-4834-a4de-3c7fa0674b23'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:34.508863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] The token cache is being invalidate because project be463be8e72c4521b0668422cb52e8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:34.509242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:34.510028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:34.512206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 967/1954] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:34 2022] DELETE /identity/v3/projects/be463be8e72c4521b0668422cb52e8cb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:34.513130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.514592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.537367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 988/1955] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:34.539257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.542227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.567182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba902-91a3-4104-9aef-63490c76a4af None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.617323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e037b-504b-4834-a4de-3c7fa0674b23 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.622513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.623211 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.623956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.624283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.626689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.646380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:34.672696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.673358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.673697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.674159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.676110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.692457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.695317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.711188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 989/1956] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:34.712132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.715004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.724879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.727448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d72339-b66a-493d-922a-3a0518329de6 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.757179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 968/1957] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:34.761896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.762176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.762447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.791456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:34.809884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:34.810895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.818451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feaaca6-bf22-4277-b85d-4fe952646371 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.825009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.826285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.826777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.827650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] PATH_INFO: `/v3/projects/3f952d858bad4635b73619655a22411c/users/5025893a6ddf4a3b936c8f2d4f47b62a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.852777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f952d858bad4635b73619655a22411c, user_id=5025893a6ddf4a3b936c8f2d4f47b62a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.855118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 969/1958] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:34.856786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.857232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:34.858255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.872284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea569642-8b3a-4889-a4e2-08e7f89bba76 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.881144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 990/1959] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:34 2022] PUT /identity/v3/projects/3f952d858bad4635b73619655a22411c/users/5025893a6ddf4a3b936c8f2d4f47b62a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:34.882832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:34.885605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.962279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.963687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.964049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.964393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.967383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:34.975799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b26a06-6963-44c8-8410-da9a6482959d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:34.982671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:34.983492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:34.984152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:34.984433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] PATH_INFO: `/v3/projects/69f342106fa745dea28aba7fd041e95c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:34.985773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f342106fa745dea28aba7fd041e95c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:34.988076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.044635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f342106fa745dea28aba7fd041e95c', 'request_id': 'req-8e67b907-5ac5-4c44-befe-9a2d270fc943'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.054847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f342106fa745dea28aba7fd041e95c', 'request_id': 'req-8e67b907-5ac5-4c44-befe-9a2d270fc943'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.064838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f342106fa745dea28aba7fd041e95c', 'request_id': 'req-8e67b907-5ac5-4c44-befe-9a2d270fc943'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.066797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.068378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937af22-31d2-434a-82a0-d82e971416ef service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.077753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] The token cache is being invalidate because project 69f342106fa745dea28aba7fd041e95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:35.078050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.078702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.079483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 970/1960] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:35.080757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.081154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.081498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.092172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 991/1961] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:34 2022] DELETE /identity/v3/projects/69f342106fa745dea28aba7fd041e95c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:35.092172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.092896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.109806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:35.109806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.158428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:35.169304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:35.179549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 971/1962] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:35.180243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.181831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.184299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b907-5ac5-4c44-befe-9a2d270fc943 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.190409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.191846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.192166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.192386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.193206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.195004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.206643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 992/1963] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:35.207665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.209830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.224359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0eac-bca1-4a37-be6f-a69bf41aa957 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.229293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.230095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.230296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.230578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] PATH_INFO: `/v3/users/2c8f19050649444eb9a7a9110f8c0538` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.231863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8f19050649444eb9a7a9110f8c0538)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.233960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.269766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0be975-5274-44f4-b94c-9e8291ac5347 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.274154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.275026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.275377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.275717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.278311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.287839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8f19050649444eb9a7a9110f8c0538'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.297449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8f19050649444eb9a7a9110f8c0538'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.297908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8f19050649444eb9a7a9110f8c0538'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.298542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8f19050649444eb9a7a9110f8c0538'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.306304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8f19050649444eb9a7a9110f8c0538'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.330550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 993/1964] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:35 2022] DELETE /identity/v3/users/2c8f19050649444eb9a7a9110f8c0538 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:35.330550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.331443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.361500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.363554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.374218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 972/1965] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:35.375126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.385937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0be77-5556-43e8-843f-b54526978a72 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.421275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078f86-14c1-4806-b1bb-015a74dcf721 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.432522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.434366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.434767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.435290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] PATH_INFO: `/v3/users/ed56f9ca03814844a13a12cfa9d8674e` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.437219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed56f9ca03814844a13a12cfa9d8674e)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.440342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.469393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.470753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.471086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.471388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.472241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:35.473486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.474697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.490344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 973/1966] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:35.491315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.496633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.514418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b8b26-6f77-4bc9-998d-2831e5f90073 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.524487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.527558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.527969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.528558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] PATH_INFO: `/v3/projects/3f952d858bad4635b73619655a22411c/users/5025893a6ddf4a3b936c8f2d4f47b62a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.533047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f952d858bad4635b73619655a22411c, user_id=5025893a6ddf4a3b936c8f2d4f47b62a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.538033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed56f9ca03814844a13a12cfa9d8674e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.538645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.549701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed56f9ca03814844a13a12cfa9d8674e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.550586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed56f9ca03814844a13a12cfa9d8674e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.551810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed56f9ca03814844a13a12cfa9d8674e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.577894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-86c1d4fc-379a-4cdb-84cd-e80e86b3a138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed56f9ca03814844a13a12cfa9d8674e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:35.585820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 974/1967] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:35 2022] PUT /identity/v3/projects/3f952d858bad4635b73619655a22411c/users/5025893a6ddf4a3b936c8f2d4f47b62a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:35.587435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.590049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:35.592363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.640163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 994/1968] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:35 2022] DELETE /identity/v3/users/ed56f9ca03814844a13a12cfa9d8674e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:35.641413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061baeed-4ced-4a04-bfad-337a1a359c47 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.641778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061baeed-4ced-4a04-bfad-337a1a359c47 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.642126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061baeed-4ced-4a04-bfad-337a1a359c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.645942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:35.679462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-061baeed-4ced-4a04-bfad-337a1a359c47 None None] MFA Rules not processed for user `5025893a6ddf4a3b936c8f2d4f47b62a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:35.681276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-061baeed-4ced-4a04-bfad-337a1a359c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.733320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4973-a0c8-4e11-a052-721669b2ff47 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.777661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 995/1969] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:35 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:35.779243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.779532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.779804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.800332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:35.801650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.814000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.814808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.815192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.815544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.816447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.818573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:35.849865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 975/1970] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:35 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:35.850893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.856462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.879206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 996/1971] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:35.880597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:35.883750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:35.907921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fde6-d144-4911-b4b2-088032bc0847 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.934766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce6b0-ce77-460f-a24c-6c4f56f90f03 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:35.938536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.939163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.939437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.939713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.994862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:35.995507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:35.996303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:35.996575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] PATH_INFO: `/v3/projects/b2406f7a61c7472dbb750bdbad5aa237` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:35.997777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2406f7a61c7472dbb750bdbad5aa237)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:35.999976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.008875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.010349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.013432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:36.021003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 976/1972] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:35 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:36.021953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.033836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca43555-5e23-4781-ae48-f2bb2cb820cb service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.039424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.040036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.040315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.040582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.041208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.043359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.053417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:36.073278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2406f7a61c7472dbb750bdbad5aa237', 'request_id': 'req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.075746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 977/1973] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/users => generated 284 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:36.077675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.080124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.081625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2406f7a61c7472dbb750bdbad5aa237', 'request_id': 'req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.096607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2406f7a61c7472dbb750bdbad5aa237', 'request_id': 'req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.115656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] The token cache is being invalidate because project b2406f7a61c7472dbb750bdbad5aa237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:36.116178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.117094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.121839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 997/1974] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:35 2022] DELETE /identity/v3/projects/b2406f7a61c7472dbb750bdbad5aa237 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:36.122960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.126786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.166245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e540-7969-482b-945b-9e6908741186 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.227655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00ec87-1aaa-43ad-a5e9-02d508b24a37 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.232456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.232770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.232973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.233507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.233829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.234422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.236225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.238089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.238445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.239215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.240902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3b8e7-5101-45a3-a717-79db6f7122e9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.249474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.250973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.252385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 998/1975] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:36.253602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.253913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.254191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.260867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 978/1976] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:36.262513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.263940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.274521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:36.275299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af78d1c-d107-48d3-b9ff-3db6f7d8e2ef service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.275736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.279989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.280615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.280891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.281218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] PATH_INFO: `/v3/users/91dca5fb81b24cac8ce9e18dc5d89f72` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.282368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dca5fb81b24cac8ce9e18dc5d89f72)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.284296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.291013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 999/1977] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:36.291013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.296697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcbdda-954c-4c89-9b95-f0cac44974d8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.301693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.302399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.302628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.302914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.303571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.305165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c5a8f-2f13-42e7-81fc-2c04566478ef admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.317182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1000/1978] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:36.318596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.318722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.318922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.342634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:36.344094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.346386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dca5fb81b24cac8ce9e18dc5d89f72'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.356736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1001/1979] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:36.356736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.357722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dca5fb81b24cac8ce9e18dc5d89f72'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.358553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dca5fb81b24cac8ce9e18dc5d89f72'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.359343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dca5fb81b24cac8ce9e18dc5d89f72'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.364920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93cb3f-8e6a-4a8f-85c0-bbc9ec9a931d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.369061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dca5fb81b24cac8ce9e18dc5d89f72'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.370520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.371789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.372061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.372330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] PATH_INFO: `/v3/projects/235c1be2b30f4270929d79c984e86a37/users/cccc5abaa14f4878a4a86e381270bf0f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.389057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=235c1be2b30f4270929d79c984e86a37, user_id=cccc5abaa14f4878a4a86e381270bf0f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.391759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575e929-597f-42ec-9e9d-30d1dbacd20b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.393259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 979/1980] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:36 2022] DELETE /identity/v3/users/91dca5fb81b24cac8ce9e18dc5d89f72 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:36.394183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.395871 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.407434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1002/1981] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:36 2022] PUT /identity/v3/projects/235c1be2b30f4270929d79c984e86a37/users/cccc5abaa14f4878a4a86e381270bf0f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:36.408455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8d7-8764-4fce-83b2-8282bcdab6a7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.408737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8d7-8764-4fce-83b2-8282bcdab6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.409002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf8d7-8764-4fce-83b2-8282bcdab6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.430266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5ceaf8d7-8764-4fce-83b2-8282bcdab6a7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:36.431411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5ceaf8d7-8764-4fce-83b2-8282bcdab6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.441228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:36.481542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:36.500026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b98d29-4c04-4cf7-b7ae-8ee6fd878a2a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.508230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1003/1982] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:36.508230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.508230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.508230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.509413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.510174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.510556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.510922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.511746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:36.512101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.514004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.533078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 980/1983] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:36.533078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.534913 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.541064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] MFA Rules not processed for user `cccc5abaa14f4878a4a86e381270bf0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:36.543616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.601832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb03a9e-d4a4-4863-a2b0-c40ab75864e4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.607183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.608246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.608809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.609205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.621663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.624049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.637429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1004/1984] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:36.637429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.637693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 981/1985] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:36.637693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.638969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.653286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87989caf-ab54-4912-b0f5-50345b8dc44b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.654360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b26ee-c9bb-4ee7-8cb6-d9abdd77b311 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.658415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.659509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.659724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.660109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.660338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.660617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.660801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] PATH_INFO: `/v3/projects/3c7a58e3c6ab48bdb36fec632121b622` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.660916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.662809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7a58e3c6ab48bdb36fec632121b622)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.664118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.666068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.666217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:36.706791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1005/1986] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/projects => generated 407 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:36.706791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.709529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.740089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7a58e3c6ab48bdb36fec632121b622', 'request_id': 'req-06992bc7-6d2a-4593-b7f2-27d4235e4c61'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.767834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7a58e3c6ab48bdb36fec632121b622', 'request_id': 'req-06992bc7-6d2a-4593-b7f2-27d4235e4c61'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.786190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7a58e3c6ab48bdb36fec632121b622', 'request_id': 'req-06992bc7-6d2a-4593-b7f2-27d4235e4c61'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.809070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] The token cache is being invalidate because project 3c7a58e3c6ab48bdb36fec632121b622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:36.809392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.810096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:36.820411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 982/1987] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:36 2022] DELETE /identity/v3/projects/3c7a58e3c6ab48bdb36fec632121b622 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:36.821581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:36.823291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:36.833143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838ba09-eee8-42d6-94be-7875d98a2d15 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.923862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:36.976216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992bc7-6d2a-4593-b7f2-27d4235e4c61 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:36.981953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:36.983334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:36.983688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:36.984032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:36.984796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:36.986948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.005272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.006301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.006932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.007207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.007855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.010845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.014193 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 983/1988] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/users => generated 304 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:37.015411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.018245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.021267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:37.034733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ee352-1075-462a-8627-cd8229b8344a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.044556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.045697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.046268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.046627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] PATH_INFO: `/v3/users/03359f27225c47d18ac5b0c776ecea68` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.048537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03359f27225c47d18ac5b0c776ecea68)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.052373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.061845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1006/1989] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:36 2022] POST /identity/v3/projects => generated 367 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:37.061845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.120476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03359f27225c47d18ac5b0c776ecea68'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.130238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03359f27225c47d18ac5b0c776ecea68'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.130994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03359f27225c47d18ac5b0c776ecea68'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.131760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03359f27225c47d18ac5b0c776ecea68'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.138945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2e6c9-a83a-4cfa-b65e-22bc0d86a9f7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.140656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03359f27225c47d18ac5b0c776ecea68'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.143770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.144561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.144673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.145163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.146027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.148134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.158535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1007/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:37.162193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.165948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 984/1991] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:37 2022] DELETE /identity/v3/users/03359f27225c47d18ac5b0c776ecea68 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:37.165948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.166139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.166213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.198661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c141-7543-4fd4-83f6-a5c06e33a03f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.258233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8b693-eb2b-4161-bda8-51b5ec42dc3b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.263221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.263858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.264136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.264407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.265113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.266848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.278687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 985/1992] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:37.280787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.282728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.298179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c6a-a822-4b22-824b-a9fafd07f0f3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.314386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.314914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.315191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.315458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.316100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.318988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.319165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.319650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.319927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.320197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] PATH_INFO: `/v3/projects/c184cbb654a44118b2b77bf5609cb459` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.325446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c184cbb654a44118b2b77bf5609cb459)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.333013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.374432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1008/1993] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:37 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:37.374432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.390989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd31cd6-60c9-4c49-bd6c-6282a093f2cc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.405420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.406141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.406417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.406687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] PATH_INFO: `/v3/projects/32bf830260544eecbe39c65b0b4f4699/users/59ef3c627a7f430a9a24abda2496e137/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.407227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:37.423395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c184cbb654a44118b2b77bf5609cb459', 'request_id': 'req-89b8762b-d306-42c0-86d0-9f09bfd0349b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.435235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c184cbb654a44118b2b77bf5609cb459', 'request_id': 'req-89b8762b-d306-42c0-86d0-9f09bfd0349b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.447619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c184cbb654a44118b2b77bf5609cb459', 'request_id': 'req-89b8762b-d306-42c0-86d0-9f09bfd0349b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.449417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bf830260544eecbe39c65b0b4f4699, user_id=59ef3c627a7f430a9a24abda2496e137, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.452234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.464088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] The token cache is being invalidate because project c184cbb654a44118b2b77bf5609cb459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:37.465324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.466637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.472043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 986/1994] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:37 2022] DELETE /identity/v3/projects/c184cbb654a44118b2b77bf5609cb459 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:37.472043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.472846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.482599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1009/1995] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:37 2022] PUT /identity/v3/projects/32bf830260544eecbe39c65b0b4f4699/users/59ef3c627a7f430a9a24abda2496e137/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:37.486526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.493274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.594502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:37.626400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c84b7-62d7-4acd-bb5b-3717f52d8d0a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.627474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8762b-d306-42c0-86d0-9f09bfd0349b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.659377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.660291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.661049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.661049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] PATH_INFO: `/v3/users/5025893a6ddf4a3b936c8f2d4f47b62a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.662484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5025893a6ddf4a3b936c8f2d4f47b62a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.664661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.664982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.665625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.665942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.666194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.666847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.677254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05951181-9d53-4c03-a9b0-a78cf4db6965 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:37.700688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 987/1996] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:37.702593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.702707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.702937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.734897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5025893a6ddf4a3b936c8f2d4f47b62a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.736097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] MFA Rules not processed for user `59ef3c627a7f430a9a24abda2496e137`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:37.737467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.746317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5025893a6ddf4a3b936c8f2d4f47b62a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.749600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5025893a6ddf4a3b936c8f2d4f47b62a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.751648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5025893a6ddf4a3b936c8f2d4f47b62a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.770464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5025893a6ddf4a3b936c8f2d4f47b62a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:37.795420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1010/1997] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:37 2022] DELETE /identity/v3/users/5025893a6ddf4a3b936c8f2d4f47b62a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:37.796041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.797598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.832093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 988/1998] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:37 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:37.837433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:37.839874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:37.858595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995c93a-4934-4b5b-b533-3658707c5072 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.920358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501287e5-2f46-4e94-a580-254d220b9f5e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:37.932007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:37.932706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:37.933844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:37.935961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] PATH_INFO: `/v3/projects/9ec4596180a8452f9557e7c8275ea87f/users/7f62b83c684a41f182b0c47312c6b66d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:37.989373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4596180a8452f9557e7c8275ea87f, user_id=7f62b83c684a41f182b0c47312c6b66d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:37.990924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:37.997347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.014372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.016048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.016418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.016923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.017983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.021065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.025788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1011/1999] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:37 2022] PUT /identity/v3/projects/9ec4596180a8452f9557e7c8275ea87f/users/7f62b83c684a41f182b0c47312c6b66d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:38.026704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.028531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.079031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 989/2000] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:37 2022] POST /identity/v3/projects => generated 407 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:38.080270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.100091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:38.140634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288daaf-873c-4571-b447-5d8d0e9192ee admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.253047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c6279-871f-4149-8079-6840328857fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.254587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac99d-84ac-4966-8ce8-279f1bd95ac9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.254707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c6279-871f-4149-8079-6840328857fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.254939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c6279-871f-4149-8079-6840328857fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.255213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c6279-871f-4149-8079-6840328857fa admin admin] PATH_INFO: `/v3/projects/4886d00185ba46d4ac095e48da1fe056` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.257014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c6279-871f-4149-8079-6840328857fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4886d00185ba46d4ac095e48da1fe056)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.261677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c6279-871f-4149-8079-6840328857fa admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.269051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.270343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.270722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.271069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.271864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.273620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.286375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 990/2001] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:38.287238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.299848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3661e6-658a-44e9-94dc-0a2975090f65 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.306856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.307630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.307948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.308774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.309526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.311861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.343179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 991/2002] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:38 2022] POST /identity/v3/users => generated 310 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:38.343179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.345146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4886d00185ba46d4ac095e48da1fe056', 'request_id': 'req-194c6279-871f-4149-8079-6840328857fa'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.355735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ab81e-4714-4963-8025-e8ffceef5802 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.356637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4886d00185ba46d4ac095e48da1fe056', 'request_id': 'req-194c6279-871f-4149-8079-6840328857fa'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.368293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4886d00185ba46d4ac095e48da1fe056', 'request_id': 'req-194c6279-871f-4149-8079-6840328857fa'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.385269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] The token cache is being invalidate because project 4886d00185ba46d4ac095e48da1fe056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:38.386916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.387891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.391328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1012/2003] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:38 2022] DELETE /identity/v3/projects/4886d00185ba46d4ac095e48da1fe056 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:38.392485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.394494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.444437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.447052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.447385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.447812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] PATH_INFO: `/v3/projects/9ec4596180a8452f9557e7c8275ea87f/users/7f62b83c684a41f182b0c47312c6b66d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.452321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec4596180a8452f9557e7c8275ea87f, user_id=7f62b83c684a41f182b0c47312c6b66d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.455726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575feba-b5b1-495f-bb94-691f30cba7bb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.481258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 992/2004] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:38 2022] PUT /identity/v3/projects/9ec4596180a8452f9557e7c8275ea87f/users/7f62b83c684a41f182b0c47312c6b66d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:38.489917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dacd-d211-44a0-b62f-41be713825f3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.489917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dacd-d211-44a0-b62f-41be713825f3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.489917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dacd-d211-44a0-b62f-41be713825f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.515335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f6f5dacd-d211-44a0-b62f-41be713825f3 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:38.517823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f6f5dacd-d211-44a0-b62f-41be713825f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.540000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c6279-871f-4149-8079-6840328857fa admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.579653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:38.619257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:38.647654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 993/2005] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:38.649104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df27-e4fe-480a-a171-f86891a6e029 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.649416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df27-e4fe-480a-a171-f86891a6e029 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.649988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350df27-e4fe-480a-a171-f86891a6e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.650366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.651255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.651602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.652007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.653357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.655137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.672604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1013/2006] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:38.673874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.676988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.683899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e350df27-e4fe-480a-a171-f86891a6e029 None None] MFA Rules not processed for user `7f62b83c684a41f182b0c47312c6b66d`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:38.685194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e350df27-e4fe-480a-a171-f86891a6e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.691776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef29a8-0d3c-42f5-8185-a83d51cf6b75 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.697677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.698363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.699320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.699604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] PATH_INFO: `/v3/projects/3f952d858bad4635b73619655a22411c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.701128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f952d858bad4635b73619655a22411c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.703080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.760664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f952d858bad4635b73619655a22411c', 'request_id': 'req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.765417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 994/2007] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:38.767227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.768020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.768561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.769202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f952d858bad4635b73619655a22411c', 'request_id': 'req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.779539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f952d858bad4635b73619655a22411c', 'request_id': 'req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.795178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] The token cache is being invalidate because project 3f952d858bad4635b73619655a22411c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:38.795465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.796307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:38.800158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:38.801495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1014/2008] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:38 2022] DELETE /identity/v3/projects/3f952d858bad4635b73619655a22411c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:38.801811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.802415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.803727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.868873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 995/2009] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:38.869828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:38.871180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.875036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c560da-fc11-4a6b-a7ae-929d0d9d9a35 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.879972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.880643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.880920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.881193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] PATH_INFO: `/v3/projects/5d0b05fce155440aa8b4195e8ec151bd/users/b9a0d94cc276405388d6b52d5504606a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.903411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0b05fce155440aa8b4195e8ec151bd, user_id=b9a0d94cc276405388d6b52d5504606a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:38.906130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75524404-fbdf-441c-b8d6-222b20a828ee admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:38.919570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1015/2010] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:38 2022] PUT /identity/v3/projects/5d0b05fce155440aa8b4195e8ec151bd/users/b9a0d94cc276405388d6b52d5504606a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:38.920569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.920879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.921151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.940013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4c5da-d342-48cf-b065-4c312a63944f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:38.941349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:38.942647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:38.966784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:38.978213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:38.991117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:38.991732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:38.992175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:38.992448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:38.994336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.007025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1016/2011] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:39.007851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.018030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b6f9b-7a52-4ceb-a9bf-dd6987631263 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.022608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.023191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.023466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.023733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.024371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:39.024667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.026273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.041369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1017/2012] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:39.041369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.048789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310cc33-3420-45fe-927f-a13ca4ff0344 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.053331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.053949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.054231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.054505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.055137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.056706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37a320-cc99-4dcd-8647-e675ee3914f5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.061293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.062750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.070206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1018/2013] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:39.071687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.071972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.072246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.073582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 996/2014] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:39.074978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.085203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ab39a-f6be-4454-8a5c-c55c3a459ce1 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.090020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.090634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.090908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.091326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.091974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.093967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.094197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:39.095546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.109462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1019/2015] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:39.109462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.120439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aaf5b-3577-4c93-93a8-9c92b5dd5a6b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.131059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.131953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.132303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.132736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] PATH_INFO: `/v3/projects/5d0b05fce155440aa8b4195e8ec151bd/users/b9a0d94cc276405388d6b52d5504606a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.134847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0b05fce155440aa8b4195e8ec151bd, user_id=b9a0d94cc276405388d6b52d5504606a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.138650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.142712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 997/2016] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:39.143646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.157848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1020/2017] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:39 2022] PUT /identity/v3/projects/5d0b05fce155440aa8b4195e8ec151bd/users/b9a0d94cc276405388d6b52d5504606a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:39.160043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.221635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a81a5-5324-46b5-8379-2ec5cf946d28 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.231715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca05543-0d81-420b-8dac-6e1099e20568 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.236949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.237560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.237867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.238426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.239064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.241299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f6005-94a5-4fc9-abb3-ff847afa6550 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.270776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1021/2018] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:39.270776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.270776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.270776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.287986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.288630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.289412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.289412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.289804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:39.290111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.292183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.299313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] MFA Rules not processed for user `b9a0d94cc276405388d6b52d5504606a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:39.299475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:39.300607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.309677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 998/2019] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:39.315716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.375406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7a724-6ae7-4e03-85c4-2fededea386d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.381762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1022/2020] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:39.382564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.384187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.384815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.385091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.385367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.387435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.395063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae55d97-d32b-4e4e-87d2-f16b70a1da9a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.400336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.401001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.401284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.401935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.402278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.404431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.406847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:39.421949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1023/2021] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:39.422833 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.439084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1155e6-65aa-42b7-9051-addfefe1af5a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.441328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.442050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.442507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.442808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.443914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.445969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.459661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.461177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.470836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 999/2022] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:39.471772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.473238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.481076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1024/2023] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:39.481076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.485655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb828b-575a-4821-a9f2-175eb784bacf service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.550647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.551305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.551586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.551857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.601651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd2efd-3af2-4303-89cd-2e165fe2e9b4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.610116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.610762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.611310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.611588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.612241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.613911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.626020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1025/2024] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:39.626809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.635489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.637518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.657162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12296a6-e0d3-41f5-9beb-d506fcd53292 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.659764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1000/2025] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:39.660713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.670029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:39.674776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.675047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.675328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.675609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.676248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.678298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.689347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daeff42-6434-43c5-b814-c95434e58b4c service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.701526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.702041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.702349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.702663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] PATH_INFO: `/v3/projects/139ecc299e604bfab0ab02c7f55a4e18/users/2a8b520933d6452cba1f631e97468f8a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.713771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1026/2026] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/users => generated 290 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:39.717205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.733232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45ac0f-8f6d-475d-8c17-7837b801ca46 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.740818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.741437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.741765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.742577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.742813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.744543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.755210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139ecc299e604bfab0ab02c7f55a4e18, user_id=2a8b520933d6452cba1f631e97468f8a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.758711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cb348-ee99-4eef-bd87-2e543bd86703 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.760895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1027/2027] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:39.765469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:39.778676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2061a-2713-42fa-84c4-fa8363d5d119 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:39.781427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1001/2028] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:39 2022] PUT /identity/v3/projects/139ecc299e604bfab0ab02c7f55a4e18/users/2a8b520933d6452cba1f631e97468f8a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:39.800080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.800359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.800635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.863280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] MFA Rules not processed for user `2a8b520933d6452cba1f631e97468f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:39.864745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:39.938704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:39.942518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:39.943737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:39.944690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:39.947752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:39.952202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:39.992597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1028/2029] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:39.998321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.030083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1002/2030] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:40.030187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df3ee-ca40-4d67-8dc5-8442a5e5ce78 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.030274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.042251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.043057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.043524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.043798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] PATH_INFO: `/v3/projects/d46605f5b4fd4f96a8a2fde9f160413b/users/f90c9a836d804481ba83e4ab692a0889/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.050453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b1e60-be4b-4d48-86cb-186b1367d26d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.057919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.060270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.061178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.061609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.062625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.063900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46605f5b4fd4f96a8a2fde9f160413b, user_id=f90c9a836d804481ba83e4ab692a0889, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.065238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.066946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c86c91-c241-40d9-ab3c-f658640a8cf1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.104296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1029/2031] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:39 2022] PUT /identity/v3/projects/d46605f5b4fd4f96a8a2fde9f160413b/users/f90c9a836d804481ba83e4ab692a0889/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:40.109448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1003/2032] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:40.116649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.117188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.117642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.118507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.176994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] MFA Rules not processed for user `f90c9a836d804481ba83e4ab692a0889`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:40.178920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:40.185700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:40.261044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ff59c-7f24-4efd-bdee-9206280a7b6b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.269680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.270554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.270861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.271189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.271847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.274424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.280363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1030/2033] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:40.280363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.303803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1004/2034] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:40.309565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.327549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960662c-80f2-4351-a946-cfcef1f4c291 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.334212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.334765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.335046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.335312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.335992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.337936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.365008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90c0e3-2c6a-40c1-b70a-705d70784828 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.369355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.370206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.370534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.370847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.372813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:40.377561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1005/2035] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:40.377561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.394349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:40.450591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.452461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.481782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1031/2036] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:40 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:40.483195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.508145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c801f-84bc-406f-ad2b-2bcd43083223 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.508630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f88e-06df-479b-9bed-404cdcd92538 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.518601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.519321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.519664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.520002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.520702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.521324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.522316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.522963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.523245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.524443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.525244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.528309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.548274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1006/2037] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:40.549461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.565195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cfb84-f46f-403c-a72a-9f22d4f4f6c3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.569386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1032/2038] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/users => generated 290 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:40.570441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.572533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.573567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.574331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.574707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] PATH_INFO: `/v3/projects/a185a7668c814f8eb1e56613da31118b/users/b4f5a7c6dc734d119df05a8dd4ff5074/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.591768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1289a-8971-4fef-8a21-2a27b05e1923 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.599688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a185a7668c814f8eb1e56613da31118b, user_id=b4f5a7c6dc734d119df05a8dd4ff5074, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.602422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.603041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.603314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.603579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.604215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.604346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.606408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.638251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1033/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:40.640603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.646581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1007/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:40 2022] PUT /identity/v3/projects/a185a7668c814f8eb1e56613da31118b/users/b4f5a7c6dc734d119df05a8dd4ff5074/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:40.651056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.662551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4ac3a-e3d7-49f7-b427-31a85670186d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.716426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:40.780560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.781207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.781435 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.781921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.783320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.787679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.789628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7f1e1-f08e-4759-a86e-49895785e3e1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.800557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.801424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.802021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.802933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.805675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.808810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.811922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1034/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:40.816403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.829249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1008/2042] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:40.830731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12614a-9d14-49be-941a-3857d91344c8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.833856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:40.836283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.837117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.837398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.837680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] PATH_INFO: `/v3/projects/cb1d8aef0ec34bd99c898db572c96178/users/90e798214e5441aea89416348b69127c/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.845815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f0b8f-9be2-444a-8255-9d338c50e250 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:40.851657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:40.852446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.852738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.853456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] PATH_INFO: `/v3/projects/a185a7668c814f8eb1e56613da31118b/users/b4f5a7c6dc734d119df05a8dd4ff5074/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.856106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a185a7668c814f8eb1e56613da31118b, user_id=b4f5a7c6dc734d119df05a8dd4ff5074, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.857654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1d8aef0ec34bd99c898db572c96178, user_id=90e798214e5441aea89416348b69127c, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:40.860083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515760c-00a4-436d-967f-498d97527fda admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.861970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905c0f5-aa18-4d16-8948-611f8d652018 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:40.878443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1009/2043] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:40 2022] PUT /identity/v3/projects/a185a7668c814f8eb1e56613da31118b/users/b4f5a7c6dc734d119df05a8dd4ff5074/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:40.895000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.895292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.896397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1035/2044] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:40 2022] PUT /identity/v3/projects/cb1d8aef0ec34bd99c898db572c96178/users/90e798214e5441aea89416348b69127c/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:40.906559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.910496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:40.910827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:40.911392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:40.972997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] MFA Rules not processed for user `90e798214e5441aea89416348b69127c`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:40.974421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:40.980577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] MFA Rules not processed for user `b4f5a7c6dc734d119df05a8dd4ff5074`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:40.981860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:41.080854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1036/2045] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:41.082566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.119155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:41.145514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1010/2046] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:41.199528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561cc05-71d4-400b-8133-fe535264f3f5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.206200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.206884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.207187 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.207460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.208106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.210161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.239746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.260682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1037/2047] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:41.262093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.296871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:41.354275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6726200-4165-4624-a1e4-16bcfea54835 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.374052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e5229-c3ba-47c1-9858-639c7d14ed70 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.379969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.381083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.381546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.381848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.382747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.384677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.423126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1038/2048] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:41 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:41.424835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.447026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a36f3-2a4c-4f9e-874a-14f13aa5e469 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.460452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.462701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.463094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.464063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.464176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.464999 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.465109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.465480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.467822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.472735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.495487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1039/2049] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:41.500311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.526702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517bc958-3d3f-4c69-ab9f-01640aec271a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.546459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.547472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.548455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.549042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] PATH_INFO: `/v3/projects/815c88539c544fe79d10dfb8afb37733/users/9bdac99562f646b0a8f83e3160532b18/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.572566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.574548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.581594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815c88539c544fe79d10dfb8afb37733, user_id=9bdac99562f646b0a8f83e3160532b18, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.584557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.613861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1011/2050] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:41.620606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1040/2051] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:41 2022] PUT /identity/v3/projects/815c88539c544fe79d10dfb8afb37733/users/9bdac99562f646b0a8f83e3160532b18/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:41.624760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.646686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eafe-ceeb-440d-8803-18af39ec98b8 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.781485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.782555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.783127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.784763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.786395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.794534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe7bd5-811d-4f6e-ae85-479dc5b7e00f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.803619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:41.813442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1012/2052] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:41.817544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:41.835624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857a1e3-4dfe-4895-85cb-b376cccfae17 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:41.858079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:41.859270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.863183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.863183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] PATH_INFO: `/v3/projects/815c88539c544fe79d10dfb8afb37733/users/9bdac99562f646b0a8f83e3160532b18/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.878638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] RBAC: Authorizing `identity:create_grant(project_id=815c88539c544fe79d10dfb8afb37733, user_id=9bdac99562f646b0a8f83e3160532b18, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:41.881300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:41.902777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1041/2053] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:41 2022] PUT /identity/v3/projects/815c88539c544fe79d10dfb8afb37733/users/9bdac99562f646b0a8f83e3160532b18/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:41.915045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:41.915972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:41.916588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:41.974314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] MFA Rules not processed for user `9bdac99562f646b0a8f83e3160532b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:41.975630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:42.079383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1013/2054] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:42.095944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:42.164912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127ca00-f0eb-4532-be7c-661c63b9acea admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:42.171247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:42.172088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:42.172480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:42.172904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:42.188087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:42.189847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:42.208699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1042/2055] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:42.256183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:42.268223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43a6d6-c818-4af4-98ce-0138716e1b3b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:42.272173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:42.272777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:42.273055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:42.273323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:42.339607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:42.341243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8a2f0-c928-411d-830f-3b5e31259acb service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:42.352888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1014/2056] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:46.596751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:46.598308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:46.677585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7daee-d296-4b6a-bf92-653fdef92674 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:46.682390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:46.683085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:46.683347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:46.683631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] PATH_INFO: `/v3/users/cccc5abaa14f4878a4a86e381270bf0f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:46.684805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccc5abaa14f4878a4a86e381270bf0f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:46.688022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:46.691116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:46.795418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccc5abaa14f4878a4a86e381270bf0f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:46.814331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccc5abaa14f4878a4a86e381270bf0f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:46.815035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccc5abaa14f4878a4a86e381270bf0f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:46.816265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccc5abaa14f4878a4a86e381270bf0f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:46.832447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccc5abaa14f4878a4a86e381270bf0f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:46.862335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1043/2057] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:46 2022] DELETE /identity/v3/users/cccc5abaa14f4878a4a86e381270bf0f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:46.875727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:46.877108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:46.877583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:46.919162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:46.920395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:47.022837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1015/2058] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:47.037819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:47.039330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:47.104481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3af37-403a-4925-a345-1a84d011d293 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:47.108304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:47.109010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:47.109353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:47.109717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:47.119578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:47.121048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:47.130767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1044/2059] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:47.422323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:47.424203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:47.441114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb6206-a34b-42ea-b60d-b2e160c692c9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:47.447786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:47.449183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:47.449550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:47.449861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] PATH_INFO: `/v3/projects/235c1be2b30f4270929d79c984e86a37` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:47.451085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=235c1be2b30f4270929d79c984e86a37)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:47.453062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:47.476843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:47.503736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235c1be2b30f4270929d79c984e86a37', 'request_id': 'req-d4ce2b96-f269-4412-8373-dc2457e161c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.510312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235c1be2b30f4270929d79c984e86a37', 'request_id': 'req-d4ce2b96-f269-4412-8373-dc2457e161c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.518420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235c1be2b30f4270929d79c984e86a37', 'request_id': 'req-d4ce2b96-f269-4412-8373-dc2457e161c2'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.529405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] The token cache is being invalidate because project 235c1be2b30f4270929d79c984e86a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:47.529695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.530475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.532132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1016/2060] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:47 2022] DELETE /identity/v3/projects/235c1be2b30f4270929d79c984e86a37 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:47.536161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:47.545903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aca547-7f4a-4150-a145-90ec39fffe55 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:47.606655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:47.607289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:47.607565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:47.607840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] PATH_INFO: `/v3/users/7f62b83c684a41f182b0c47312c6b66d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:47.608971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f62b83c684a41f182b0c47312c6b66d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:47.610963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:47.641178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:47.664170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f62b83c684a41f182b0c47312c6b66d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.673294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f62b83c684a41f182b0c47312c6b66d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.674000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f62b83c684a41f182b0c47312c6b66d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.674874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f62b83c684a41f182b0c47312c6b66d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.682476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f62b83c684a41f182b0c47312c6b66d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:47.702344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1045/2061] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:47 2022] DELETE /identity/v3/users/7f62b83c684a41f182b0c47312c6b66d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:48.003771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.006100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.068447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce2b96-f269-4412-8373-dc2457e161c2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.131248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.132519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.147123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.149771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.150090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.151231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.153473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.229128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.230682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.235977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ed5f9-e60c-4704-9dd6-4b85043836c8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.240286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1017/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:48.240674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.241160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.241442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.241790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.242080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] PATH_INFO: `/v3/projects/9ec4596180a8452f9557e7c8275ea87f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.242531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.243507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec4596180a8452f9557e7c8275ea87f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.245985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.255910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049be06-ea98-4893-9def-1a1827658f38 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.260928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.261632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.261961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.262228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] PATH_INFO: `/v3/users/f90c9a836d804481ba83e4ab692a0889` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.263409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90c9a836d804481ba83e4ab692a0889)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.265303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.271974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:48.299266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec4596180a8452f9557e7c8275ea87f', 'request_id': 'req-a879338f-322c-49d5-8f90-5c2232fd73c1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.305871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec4596180a8452f9557e7c8275ea87f', 'request_id': 'req-a879338f-322c-49d5-8f90-5c2232fd73c1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.316342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec4596180a8452f9557e7c8275ea87f', 'request_id': 'req-a879338f-322c-49d5-8f90-5c2232fd73c1'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.323838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90c9a836d804481ba83e4ab692a0889'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.332653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90c9a836d804481ba83e4ab692a0889'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.333475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90c9a836d804481ba83e4ab692a0889'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.334109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90c9a836d804481ba83e4ab692a0889'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.335866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] The token cache is being invalidate because project 9ec4596180a8452f9557e7c8275ea87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:48.336201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.340756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a879338f-322c-49d5-8f90-5c2232fd73c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.341382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1046/2063] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:48 2022] DELETE /identity/v3/projects/9ec4596180a8452f9557e7c8275ea87f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:48.344990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ac8e504-1a8a-441c-88d6-86b0c17b730b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90c9a836d804481ba83e4ab692a0889'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:48.346163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b1867-fa2f-4eb6-b1d8-3c6e433d383a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.347148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b1867-fa2f-4eb6-b1d8-3c6e433d383a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.347521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b1867-fa2f-4eb6-b1d8-3c6e433d383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.368124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1018/2064] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:48 2022] DELETE /identity/v3/users/f90c9a836d804481ba83e4ab692a0889 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:48.371155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-840b1867-fa2f-4eb6-b1d8-3c6e433d383a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:48.372379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-840b1867-fa2f-4eb6-b1d8-3c6e433d383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.373003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.373441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.373817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.411294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:48.412526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.417395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:48.465993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1047/2065] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:48.474735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.475087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.475421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.496470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:48.497678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.509768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1048/2066] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:48.509936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1019/2067] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:48.512010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.517895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.519273 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.521628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a04b1f-762f-4180-a0e0-0c1df2fee736 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.528388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.529969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.530258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.530574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.543061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:48.543682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.558493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.571816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1049/2068] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:48.577060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.598278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac90b23-2ab4-4776-877d-308391054bb2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.606066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.606843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.607484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.607858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.608618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.610041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c4b0e-21b3-4ead-a10c-ecad22924258 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.611156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.614942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.616075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.616616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.617036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.631695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.633265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.644154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1020/2069] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:48.657440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1050/2070] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:48.662143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.672716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34ded8-dffd-41ba-95f6-987f21e5b2f9 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.704208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:48.740251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.740967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.741318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.741660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.742418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.744507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.767296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1021/2071] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:48.771498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.782920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f021dcf-85ae-4379-9338-8d18c87cdaba admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.788355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.788958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.789235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.789503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.790206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.791800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.803268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1051/2072] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:48.807875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.819818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db455e-c001-49e8-9644-d6131e21f480 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.838396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.838730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.839010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.839817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.840475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.842721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597360ea-2a74-4429-a657-3aa19006daca admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.854811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1022/2073] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:48.859586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.869904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4e634-3172-4265-8010-e4a679fbe721 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:48.874890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:48.875502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.875780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.876221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] PATH_INFO: `/v3/projects/3ce9254815fa423ea3a991d9e1f727a7/users/b4ddce86d247499ab9723cb49d0d2c01/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.894214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce9254815fa423ea3a991d9e1f727a7, user_id=b4ddce86d247499ab9723cb49d0d2c01, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:48.895855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:48.913771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1052/2074] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:48 2022] PUT /identity/v3/projects/3ce9254815fa423ea3a991d9e1f727a7/users/b4ddce86d247499ab9723cb49d0d2c01/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:48.924543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:48.924836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:48.925415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:48.955565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] MFA Rules not processed for user `b4ddce86d247499ab9723cb49d0d2c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:48.957025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:48.994143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:48.994143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.050143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1023/2075] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:49.064617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.066900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.078077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270cf06-174a-44c8-86b6-ffdd3d6bb7c5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.084492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.085286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.085765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.086217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] PATH_INFO: `/v3/projects/d46605f5b4fd4f96a8a2fde9f160413b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.087703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46605f5b4fd4f96a8a2fde9f160413b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.089815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.137595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e3cee-11fe-4f41-b337-b71d3020fc44 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.141511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46605f5b4fd4f96a8a2fde9f160413b', 'request_id': 'req-ef36c463-a90e-445d-814f-ab35c540b1fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.148233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46605f5b4fd4f96a8a2fde9f160413b', 'request_id': 'req-ef36c463-a90e-445d-814f-ab35c540b1fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.152586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:49.157018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46605f5b4fd4f96a8a2fde9f160413b', 'request_id': 'req-ef36c463-a90e-445d-814f-ab35c540b1fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.169458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] The token cache is being invalidate because project d46605f5b4fd4f96a8a2fde9f160413b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:49.169764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.170431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.172294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1053/2076] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:48 2022] DELETE /identity/v3/projects/d46605f5b4fd4f96a8a2fde9f160413b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:49.176130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.177466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.189815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.190475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.190710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.190981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.192795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.200032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:49.258777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c463-a90e-445d-814f-ab35c540b1fb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.259963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.262576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.264613 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.265252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.265531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.265842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] PATH_INFO: `/v3/users/90e798214e5441aea89416348b69127c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.267039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e798214e5441aea89416348b69127c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.272648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1024/2077] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:49.299917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.302042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.317321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e180de-cd75-410a-a9d1-d9f27dc69970 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.320096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e798214e5441aea89416348b69127c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.323417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.324085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.324306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.324862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] PATH_INFO: `/v3/users/b4ddce86d247499ab9723cb49d0d2c01` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.326491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ddce86d247499ab9723cb49d0d2c01)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.328908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.329841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e798214e5441aea89416348b69127c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.330735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e798214e5441aea89416348b69127c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.331485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e798214e5441aea89416348b69127c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.341407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a9d2e4e-f591-4952-9002-6fd795a0b759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e798214e5441aea89416348b69127c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.365843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1054/2078] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:49 2022] DELETE /identity/v3/users/90e798214e5441aea89416348b69127c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:49.382125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ddce86d247499ab9723cb49d0d2c01'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.390679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ddce86d247499ab9723cb49d0d2c01'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.391404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ddce86d247499ab9723cb49d0d2c01'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.392026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ddce86d247499ab9723cb49d0d2c01'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.399282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ddce86d247499ab9723cb49d0d2c01'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.418605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1025/2079] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:49 2022] DELETE /identity/v3/users/b4ddce86d247499ab9723cb49d0d2c01 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:49.423066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.423540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.423975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.451053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:49.452269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.525039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1055/2080] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:49.531825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.533254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.610915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3b7cd-4a44-45b5-8059-ca1c1e0b323d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.615128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.615751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.616031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.616343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.630408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.632703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.653521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1026/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:49.678212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.679679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.708886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72813db5-e659-4c32-9553-3a8bea029ba6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.714602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.715347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.715737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.716105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] PATH_INFO: `/v3/projects/cb1d8aef0ec34bd99c898db572c96178` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.717392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1d8aef0ec34bd99c898db572c96178)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.724618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.724618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:49.726138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.727650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.749509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c500-7514-45b9-9645-711378927510 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:49.758140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:49.759731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.760021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.760295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] PATH_INFO: `/v3/users/59ef3c627a7f430a9a24abda2496e137` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.764120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ef3c627a7f430a9a24abda2496e137)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:49.766468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:49.788882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:49.809070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1d8aef0ec34bd99c898db572c96178', 'request_id': 'req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.820635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1d8aef0ec34bd99c898db572c96178', 'request_id': 'req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.843860 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ef3c627a7f430a9a24abda2496e137'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.849511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1d8aef0ec34bd99c898db572c96178', 'request_id': 'req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.866982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ef3c627a7f430a9a24abda2496e137'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.867686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] The token cache is being invalidate because project cb1d8aef0ec34bd99c898db572c96178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:49.868324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ef3c627a7f430a9a24abda2496e137'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.868965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.869591 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ef3c627a7f430a9a24abda2496e137'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.872027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.874223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1056/2082] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:49 2022] DELETE /identity/v3/projects/cb1d8aef0ec34bd99c898db572c96178 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:49.878153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:49.879844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.886681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8648e62d-1bcf-468a-aafb-c8db17fec0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ef3c627a7f430a9a24abda2496e137'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:49.929721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1027/2083] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:49 2022] DELETE /identity/v3/users/59ef3c627a7f430a9a24abda2496e137 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:49.931311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:49.931607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:49.932251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:49.962957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:49.964736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:49.987881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df102a-6a3c-4b6a-9cec-6678cf6fd92c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.066788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1028/2084] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:50.076407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.078667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.095197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.095778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.096058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.096327 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] PATH_INFO: `/v3/users/9bdac99562f646b0a8f83e3160532b18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.097501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdac99562f646b0a8f83e3160532b18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.099789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.103507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03236251-57fc-4bc8-9831-1726f4a64851 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.123577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.124221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.124499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.124769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] PATH_INFO: `/v3/projects/3ce9254815fa423ea3a991d9e1f727a7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.126827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce9254815fa423ea3a991d9e1f727a7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.128977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.156377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:50.181676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdac99562f646b0a8f83e3160532b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.191071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdac99562f646b0a8f83e3160532b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.192108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdac99562f646b0a8f83e3160532b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.192778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdac99562f646b0a8f83e3160532b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.200041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce9254815fa423ea3a991d9e1f727a7', 'request_id': 'req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.203099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdac99562f646b0a8f83e3160532b18'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.207718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce9254815fa423ea3a991d9e1f727a7', 'request_id': 'req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.216233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce9254815fa423ea3a991d9e1f727a7', 'request_id': 'req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.225149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1057/2085] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:49 2022] DELETE /identity/v3/users/9bdac99562f646b0a8f83e3160532b18 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:50.226590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.228251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.228376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] The token cache is being invalidate because project 3ce9254815fa423ea3a991d9e1f727a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:50.228570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.229284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8c1ada8b-ce9d-4ed0-be81-62fc0b039702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.231486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1029/2086] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:50 2022] DELETE /identity/v3/projects/3ce9254815fa423ea3a991d9e1f727a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:50.237221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed569b9-4b68-4fac-9b2f-98d3d18d2be0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.237575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed569b9-4b68-4fac-9b2f-98d3d18d2be0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.238015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed569b9-4b68-4fac-9b2f-98d3d18d2be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.269052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7ed569b9-4b68-4fac-9b2f-98d3d18d2be0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:50.270567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7ed569b9-4b68-4fac-9b2f-98d3d18d2be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.292044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:50.344594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bd79c-98ca-4115-8ffb-7b53c1bc89f8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.350566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.351353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.351718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.352115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.355221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.371155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1030/2087] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:50.382132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.382365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.382572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.417022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:50.418883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.428342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1031/2088] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:50.433526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.447761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec1769-c4db-46a3-9af4-b62a72ff84c4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.462279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.463237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.463375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.463579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.465341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:50.465650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.468978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.471176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.483033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.507442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1058/2089] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:50.508213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1032/2090] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:50.514122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.523892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095636a-bc3f-44d8-8302-f5c7857f4ba8 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.528551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.529269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.529549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.529846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.530486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.532454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.568655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1059/2091] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:50 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:50.569966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.581408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183ee2-1d14-432f-9df4-0b06b80cc61c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.645726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.646866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.647213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.647554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.648291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.650297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.673188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1033/2092] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:50 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:50.677634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.689332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7248527-3946-4940-a60b-835bc4fd2a7b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.694116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.694719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.694996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.695264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.696061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.697877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.709402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1060/2093] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:50.713472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.724521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45f93e-0c01-40e9-9223-d87eac7a44bf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.734588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.735735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.736669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.737566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.738672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.741226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.759487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1034/2094] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:50.760473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.764128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.767363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.772565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c7206-57f0-4afb-9049-8e9a6eb949b7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.777775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.779149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.779857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.780415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] PATH_INFO: `/v3/projects/277cef8137f043d6b749f1eb6e409a1e/users/fc1636727e6044ec854a4cee119d5bf8/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.798123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277cef8137f043d6b749f1eb6e409a1e, user_id=fc1636727e6044ec854a4cee119d5bf8, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.799169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c9064-80df-49ba-8c23-f78ca8b0dfab admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:50.802123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca034bb-797e-4ca1-9d06-dd6a9a36e530 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.804689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:50.805428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.805796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.806137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] PATH_INFO: `/v3/projects/815c88539c544fe79d10dfb8afb37733` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.807487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815c88539c544fe79d10dfb8afb37733)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:50.810880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:50.824537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1061/2095] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:50 2022] PUT /identity/v3/projects/277cef8137f043d6b749f1eb6e409a1e/users/fc1636727e6044ec854a4cee119d5bf8/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:50.834360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:50.834725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:50.835092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:50.856144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:50.866936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] MFA Rules not processed for user `fc1636727e6044ec854a4cee119d5bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:50.868827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.880444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815c88539c544fe79d10dfb8afb37733', 'request_id': 'req-b1307a16-1279-4259-a7c7-70aed1844425'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.893458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815c88539c544fe79d10dfb8afb37733', 'request_id': 'req-b1307a16-1279-4259-a7c7-70aed1844425'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.910757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815c88539c544fe79d10dfb8afb37733', 'request_id': 'req-b1307a16-1279-4259-a7c7-70aed1844425'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.923427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:50.928070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] The token cache is being invalidate because project 815c88539c544fe79d10dfb8afb37733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:50.929017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.931121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:50.935181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1035/2096] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:50 2022] DELETE /identity/v3/projects/815c88539c544fe79d10dfb8afb37733 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:50.984397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:50.986555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:50.998454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1062/2097] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:51.053710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:51.055726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.125570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1307a16-1279-4259-a7c7-70aed1844425 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.134360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.135531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.136161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.136589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] PATH_INFO: `/v3/projects/32bf830260544eecbe39c65b0b4f4699` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.139121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bf830260544eecbe39c65b0b4f4699)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.142037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10663f-184a-4d3a-9c72-51f74527b6e3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.143580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.146624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.147728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.147851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.148216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.161966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.164635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.179247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1063/2098] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:51.197247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:51.198911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.219644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1baa1-1760-454b-be90-34b3b35e948d service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.234202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bf830260544eecbe39c65b0b4f4699', 'request_id': 'req-edac8ee3-c073-4b23-aa25-c96125913135'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.245107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bf830260544eecbe39c65b0b4f4699', 'request_id': 'req-edac8ee3-c073-4b23-aa25-c96125913135'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.250323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bf830260544eecbe39c65b0b4f4699', 'request_id': 'req-edac8ee3-c073-4b23-aa25-c96125913135'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.261075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] The token cache is being invalidate because project 32bf830260544eecbe39c65b0b4f4699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:51.261930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.262573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.264054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1036/2099] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:50 2022] DELETE /identity/v3/projects/32bf830260544eecbe39c65b0b4f4699 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:51.267898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:51.269164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.288711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.289338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.289615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.289916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] PATH_INFO: `/v3/users/fc1636727e6044ec854a4cee119d5bf8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.291045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1636727e6044ec854a4cee119d5bf8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.292977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.336499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac8ee3-c073-4b23-aa25-c96125913135 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.337686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:51.341149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1636727e6044ec854a4cee119d5bf8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.341788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.342422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.342701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.342982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] PATH_INFO: `/v3/users/b9a0d94cc276405388d6b52d5504606a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.344264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a0d94cc276405388d6b52d5504606a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.346227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.349590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1636727e6044ec854a4cee119d5bf8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.350319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1636727e6044ec854a4cee119d5bf8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.350920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1636727e6044ec854a4cee119d5bf8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.358774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-bcf15e65-f58b-4962-bfaa-ce6954d51a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1636727e6044ec854a4cee119d5bf8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.378743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1064/2100] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:51 2022] DELETE /identity/v3/users/fc1636727e6044ec854a4cee119d5bf8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:51.383766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.384073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.384354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.384761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:51.400932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a0d94cc276405388d6b52d5504606a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.404191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:51.405549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.408308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a0d94cc276405388d6b52d5504606a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.409009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a0d94cc276405388d6b52d5504606a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.409709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a0d94cc276405388d6b52d5504606a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.416993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a0d94cc276405388d6b52d5504606a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.438041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1037/2101] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:51 2022] DELETE /identity/v3/users/b9a0d94cc276405388d6b52d5504606a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:51.474488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1065/2102] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:51.484976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:51.486428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.554239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcc7cd-96d4-4231-9d0b-50072e80939b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.558808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.559419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.559692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.559958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.643055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.645607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344654bc-b320-4f5e-a343-6646d37b06cf service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.665582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1038/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:51.738128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:51.740089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:51.769358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942349e-3573-4072-9197-1b23fa8c9f79 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:51.777875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:51.778477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.778756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.779058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] PATH_INFO: `/v3/projects/5d0b05fce155440aa8b4195e8ec151bd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.781002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0b05fce155440aa8b4195e8ec151bd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:51.783453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:51.868481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0b05fce155440aa8b4195e8ec151bd', 'request_id': 'req-d3d86ce9-e873-4338-9676-b2daa9165b05'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.877448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0b05fce155440aa8b4195e8ec151bd', 'request_id': 'req-d3d86ce9-e873-4338-9676-b2daa9165b05'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.910700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0b05fce155440aa8b4195e8ec151bd', 'request_id': 'req-d3d86ce9-e873-4338-9676-b2daa9165b05'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.924129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] The token cache is being invalidate because project 5d0b05fce155440aa8b4195e8ec151bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:51.924418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.925053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d3d86ce9-e873-4338-9676-b2daa9165b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:51.929812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1066/2104] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:51 2022] DELETE /identity/v3/projects/5d0b05fce155440aa8b4195e8ec151bd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:51.931674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:51.931960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:51.932241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:51.958308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:51.959637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.027150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:52.043977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1039/2105] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:52.047661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.048296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.048296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.077689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.078872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:52.080117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.080601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.092148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1067/2106] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:52.097404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.106509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2346b-8a81-42e6-a352-11c6dbaf756a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.113203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.113203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.113724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.114178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] PATH_INFO: `/v3/projects/277cef8137f043d6b749f1eb6e409a1e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.116233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=277cef8137f043d6b749f1eb6e409a1e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.120220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.123926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3af5c-e0c6-4e8f-836e-a73b4f86a5a7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.130106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.130836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.131116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.131420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.132144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:52.132691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.134428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.156983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1068/2107] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:52.158815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.169243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44ab18-bd16-42c7-99b4-3df94f98549c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.174323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.174914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.175188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.175456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.176076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.178185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.188184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277cef8137f043d6b749f1eb6e409a1e', 'request_id': 'req-9d069634-6b82-4d07-be3a-660c756c43ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:52.195274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277cef8137f043d6b749f1eb6e409a1e', 'request_id': 'req-9d069634-6b82-4d07-be3a-660c756c43ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:52.203788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277cef8137f043d6b749f1eb6e409a1e', 'request_id': 'req-9d069634-6b82-4d07-be3a-660c756c43ad'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:52.206369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1069/2108] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:52.210653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.214929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] The token cache is being invalidate because project 277cef8137f043d6b749f1eb6e409a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:52.215216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:52.215717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:52.215869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9d069634-6b82-4d07-be3a-660c756c43ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:52.217528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1040/2109] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:52 2022] DELETE /identity/v3/projects/277cef8137f043d6b749f1eb6e409a1e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:52.222577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.223269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.223582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.245272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:52.246514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.289452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d33ef-18bc-4925-a059-0e28bb5115f6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.295866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.296548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.296981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.297315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.298256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.300249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409b765-4f2f-4afd-89e7-95c1f9385fe3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.322463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1041/2110] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:52.325822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1070/2111] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:52.330017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.330496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.330618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.331696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.332081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.345562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a0155-01e2-4c2a-aa6e-2e4412453c2b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.351178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.351841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.352140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.352415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.353173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.354384 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:52.354952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.355631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.364167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1071/2112] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:52.365618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1042/2113] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:52.369721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.369865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.381188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee7599-c4df-435a-bc21-2ae8dbdd7289 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.381695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767a1f-f7ed-41ac-b80a-c8e9223bd4e5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.401216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.401216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.401216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.401216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.401643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.402396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:52.402396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.402560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.402560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.402560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.402560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.402850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.402932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.405768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1072/2114] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:52.410027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.415276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1043/2115] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:52.416256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.421055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352eda1-5bf7-4d9c-aa3a-e1020107ad95 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.426011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.426643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.426929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.427167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d0591-e1fb-4954-97f1-6fb2c33564b6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.427268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.427828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.429786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.432338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.433654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.434011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.434298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] PATH_INFO: `/v3/projects/0e6ac89b61e7496082403f861d98303a/users/fcae6377710b4dddb53ff100169c57e8/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.451169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6ac89b61e7496082403f861d98303a, user_id=fcae6377710b4dddb53ff100169c57e8, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.454138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e75f0-4d46-470c-8933-6ae539b18c4c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.459067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1073/2116] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:52.463630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.468196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1044/2117] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:52 2022] PUT /identity/v3/projects/0e6ac89b61e7496082403f861d98303a/users/fcae6377710b4dddb53ff100169c57e8/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:52.478906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.479317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.479683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.512510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] MFA Rules not processed for user `fcae6377710b4dddb53ff100169c57e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:52.514009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.532130 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc069-0847-496c-a573-2b96e90bfcdc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.542919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:52.590735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1045/2118] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:52.592029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:52.595570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.600727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.601808 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.602492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.602905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.603836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.605765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.607469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d996c35-a5fa-4a5c-9be5-51af05e8deef None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.612212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.612895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.613244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.613516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.614399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.617546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.633529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1074/2119] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:52.636889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.647388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5d6d-c855-43e9-b1eb-067d7bdd2337 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.652214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1046/2120] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:52.652982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.653641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.653961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.654236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.655100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.656182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.656858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.669768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1075/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:52.672688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.683635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba2c7-2c12-4ba8-b77f-fca50a667ce1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.723730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b1852-f8dd-4f42-98e0-e49ab4769e03 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.728566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.729249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.729531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.729935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.730573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.732505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.748312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.749143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.749257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.749525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.750230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.752015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.757388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1047/2122] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:52.758464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.763300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1076/2123] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:52.767262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.768817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969cda4-be62-4c94-9967-873cb6186d34 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.773386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.774003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.774280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.774550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.775186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.776834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.777129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4673-375b-415c-ac30-6ea029d0879e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.781895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.782540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.782818 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.783089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] PATH_INFO: `/v3/projects/743315b0ded543eb8848996e6f720a17/users/dce7b291f44b4b33b1727d5da994a2b8/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.787753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1048/2124] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:52.791806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.797868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743315b0ded543eb8848996e6f720a17, user_id=dce7b291f44b4b33b1727d5da994a2b8, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.800508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7110e-2f5c-4e78-aa35-9b90606bc537 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.801582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2bb97-1b6e-4d2a-a455-883f924f1c87 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.806828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.807480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.808329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.808610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] PATH_INFO: `/v3/projects/bf371b25552144a1b91eba389c49b992/users/e071d4113c214f10b34faea61389f867/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.814287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1077/2125] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:52 2022] PUT /identity/v3/projects/743315b0ded543eb8848996e6f720a17/users/dce7b291f44b4b33b1727d5da994a2b8/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:52.824416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf371b25552144a1b91eba389c49b992, user_id=e071d4113c214f10b34faea61389f867, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.825597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.825906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.826183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.827154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.841748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1049/2126] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:52 2022] PUT /identity/v3/projects/bf371b25552144a1b91eba389c49b992/users/e071d4113c214f10b34faea61389f867/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:52.845651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.856041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] MFA Rules not processed for user `dce7b291f44b4b33b1727d5da994a2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:52.857235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.911612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959de0b-b0bd-420a-8d77-181bc755207a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.916818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.917464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.917806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.918052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.918698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.920295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.921545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:52.932281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1078/2127] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:52.934030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1050/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:52.935556 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.942594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:52.944384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:52.946192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb81d9b-a1f3-44ba-9067-279329e7113f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:52.951571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:52.952223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.952502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.952776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] PATH_INFO: `/v3/projects/bf371b25552144a1b91eba389c49b992/users/e071d4113c214f10b34faea61389f867/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:52.955001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf371b25552144a1b91eba389c49b992, user_id=e071d4113c214f10b34faea61389f867, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:52.957667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190be145-76fd-4284-b120-ba07c5c043a0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:52.969134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1079/2129] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:52 2022] PUT /identity/v3/projects/bf371b25552144a1b91eba389c49b992/users/e071d4113c214f10b34faea61389f867/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:52.981616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453599-8604-40fd-9a20-36dd13795468 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:52.982002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453599-8604-40fd-9a20-36dd13795468 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:52.982349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72453599-8604-40fd-9a20-36dd13795468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.004413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de568f9-25c3-46a3-8406-6fde9cf4dc06 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.010734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-72453599-8604-40fd-9a20-36dd13795468 None None] MFA Rules not processed for user `e071d4113c214f10b34faea61389f867`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:53.012469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72453599-8604-40fd-9a20-36dd13795468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.024181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:53.060004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.060655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.060936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.061206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.090162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1080/2130] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:53.103023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453599-8604-40fd-9a20-36dd13795468 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.104289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-72453599-8604-40fd-9a20-36dd13795468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.118497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72453599-8604-40fd-9a20-36dd13795468 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.123454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.124221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.124555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.124826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.125949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.127593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.137055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.138569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.138905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1051/2131] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:53.149191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1081/2132] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:53.317993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.327564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541e8b5-08d1-4e73-a90b-d3a852454cd6 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.331305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.331909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.332214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.332584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.410380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.411918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b2fd2-68d9-42a1-9172-55fd92d8213d service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.421895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1052/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:53.490120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.491744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.598766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0a2de-eaec-4e36-82c0-d9b9f5cf6dcc service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.608544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.609532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.610160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.610960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] PATH_INFO: `/v3/users/2a8b520933d6452cba1f631e97468f8a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.612330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8b520933d6452cba1f631e97468f8a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.615491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.688843 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8b520933d6452cba1f631e97468f8a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.699964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8b520933d6452cba1f631e97468f8a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.700773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8b520933d6452cba1f631e97468f8a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.701501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8b520933d6452cba1f631e97468f8a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.709883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8b520933d6452cba1f631e97468f8a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.729872 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1082/2134] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:53 2022] DELETE /identity/v3/users/2a8b520933d6452cba1f631e97468f8a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:53.734704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.735061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.735461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.744473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:53.756825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:53.757252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.758286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.758822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.782158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:53.824906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1053/2135] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:53.829082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.830434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.836968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e309bb8-2c32-4649-8c15-a06c034c5d5e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.841745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.842391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.842521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95fb22-1802-4bac-b8b9-9bd21f9cb518 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:53.842669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.842939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] PATH_INFO: `/v3/users/ab716cb3074b486aa32ecbe078c0f80a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.844155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab716cb3074b486aa32ecbe078c0f80a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.846209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.847258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:53.847874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.848152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.848424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] PATH_INFO: `/v3/users/fcae6377710b4dddb53ff100169c57e8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.849571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcae6377710b4dddb53ff100169c57e8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:53.851499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:53.896478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab716cb3074b486aa32ecbe078c0f80a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.901549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcae6377710b4dddb53ff100169c57e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.905757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab716cb3074b486aa32ecbe078c0f80a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.906776 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab716cb3074b486aa32ecbe078c0f80a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.907303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab716cb3074b486aa32ecbe078c0f80a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.910651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcae6377710b4dddb53ff100169c57e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.911478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcae6377710b4dddb53ff100169c57e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.912447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcae6377710b4dddb53ff100169c57e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.915145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab716cb3074b486aa32ecbe078c0f80a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.922214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-78bc15c3-e6d4-4558-8529-dc3b1122540d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcae6377710b4dddb53ff100169c57e8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:53.937887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1083/2136] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:53 2022] DELETE /identity/v3/users/ab716cb3074b486aa32ecbe078c0f80a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:53.941968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:53.943816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:53.945337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1054/2137] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:53 2022] DELETE /identity/v3/users/fcae6377710b4dddb53ff100169c57e8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:53.947043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7af8de-2f89-406e-bdcf-9a1be24da3c5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:53.947619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7af8de-2f89-406e-bdcf-9a1be24da3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:53.948234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7af8de-2f89-406e-bdcf-9a1be24da3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:53.969520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ca7af8de-2f89-406e-bdcf-9a1be24da3c5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:53.970780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca7af8de-2f89-406e-bdcf-9a1be24da3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.020286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35335-2640-41ef-8124-34bacdf50401 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:54.053454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1055/2138] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:54.055455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.056614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.056900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.079735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:54.081121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.085283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:54.086970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.086970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.087512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.089316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1056/2139] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:54.090408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.090693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.092309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.105056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e73191-a79c-407f-a812-e0865631a9c7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:54.107043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:54.109616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:54.110275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.110551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.110816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.111253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:54.112764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:54.121872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:54.123247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:54.125626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1084/2140] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:54.126560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.127848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:54.136530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979b79-f1ff-46bb-b1d6-cf865ffcdada service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:54.137474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1057/2141] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:54.141323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:54.141987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.142267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.142550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.152421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:54.153892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:54.166629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1085/2142] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:54.701936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.703782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.751600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e29-f3e8-4437-8d56-c92d37d1a38f service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:54.762951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:54.763222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.763462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.763720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] PATH_INFO: `/v3/projects/ff2ad7457a0e4776a12533624801bd21` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.764993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2ad7457a0e4776a12533624801bd21)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:54.767543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:54.789751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.791238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.807914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f179eb-8fa9-4566-935f-7d56053de69e service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:54.832322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:54.834230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:54.836184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:54.836899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] PATH_INFO: `/v3/projects/139ecc299e604bfab0ab02c7f55a4e18` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:54.840480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=139ecc299e604bfab0ab02c7f55a4e18)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:54.844176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:54.882850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2ad7457a0e4776a12533624801bd21', 'request_id': 'req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.891573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2ad7457a0e4776a12533624801bd21', 'request_id': 'req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.911115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2ad7457a0e4776a12533624801bd21', 'request_id': 'req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.925622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] The token cache is being invalidate because project ff2ad7457a0e4776a12533624801bd21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:54.925946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.926600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.931120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1058/2143] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:54 2022] DELETE /identity/v3/projects/ff2ad7457a0e4776a12533624801bd21 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:54.931120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.931120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:54.946917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139ecc299e604bfab0ab02c7f55a4e18', 'request_id': 'req-749c7447-bde6-4cbb-83e1-08107c533905'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.957989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139ecc299e604bfab0ab02c7f55a4e18', 'request_id': 'req-749c7447-bde6-4cbb-83e1-08107c533905'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.967578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139ecc299e604bfab0ab02c7f55a4e18', 'request_id': 'req-749c7447-bde6-4cbb-83e1-08107c533905'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.982112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] The token cache is being invalidate because project 139ecc299e604bfab0ab02c7f55a4e18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:54.982534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.983676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:54.985343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1086/2144] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:54 2022] DELETE /identity/v3/projects/139ecc299e604bfab0ab02c7f55a4e18 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:54.986376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:54.987791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:55.012442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74ddf-ce6a-4513-9a3c-fd6e3a0f9e76 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.064063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c7447-bde6-4cbb-83e1-08107c533905 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.068725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.069353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.069626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.069929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] PATH_INFO: `/v3/users/30c72ca127dc4b30b1b46402662c9b3d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.071084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c72ca127dc4b30b1b46402662c9b3d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.073253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.074855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.075559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.075929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.076272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] PATH_INFO: `/v3/projects/0e6ac89b61e7496082403f861d98303a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.077476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6ac89b61e7496082403f861d98303a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.079502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.119395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:55.131577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c72ca127dc4b30b1b46402662c9b3d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.140349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c72ca127dc4b30b1b46402662c9b3d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.140484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6ac89b61e7496082403f861d98303a', 'request_id': 'req-798a186b-f6cd-450e-b699-49cc41ea8702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.141176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c72ca127dc4b30b1b46402662c9b3d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.141926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c72ca127dc4b30b1b46402662c9b3d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.148132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6ac89b61e7496082403f861d98303a', 'request_id': 'req-798a186b-f6cd-450e-b699-49cc41ea8702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.154307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c72ca127dc4b30b1b46402662c9b3d'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.158174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:55.160539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6ac89b61e7496082403f861d98303a', 'request_id': 'req-798a186b-f6cd-450e-b699-49cc41ea8702'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.173433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] The token cache is being invalidate because project 0e6ac89b61e7496082403f861d98303a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:55.173722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.174933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.176869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1059/2145] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:54 2022] DELETE /identity/v3/projects/0e6ac89b61e7496082403f861d98303a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:55.177461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:55.178757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:55.184977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1087/2146] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:54 2022] DELETE /identity/v3/users/30c72ca127dc4b30b1b46402662c9b3d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:55.186447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:55.188161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:55.287758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a186b-f6cd-450e-b699-49cc41ea8702 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.293245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b52e6f-4d29-4021-8181-a1211bf2ffb3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.293399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.294108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.294391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.294666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] PATH_INFO: `/v3/users/b4f5a7c6dc734d119df05a8dd4ff5074` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.295829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f5a7c6dc734d119df05a8dd4ff5074)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.297808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.305347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.306035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.306317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.306734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] PATH_INFO: `/v3/users/e071d4113c214f10b34faea61389f867` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.310620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e071d4113c214f10b34faea61389f867)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.310620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.380277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f5a7c6dc734d119df05a8dd4ff5074'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.391696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f5a7c6dc734d119df05a8dd4ff5074'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.392455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f5a7c6dc734d119df05a8dd4ff5074'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.393221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f5a7c6dc734d119df05a8dd4ff5074'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.393958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e071d4113c214f10b34faea61389f867'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.402760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f5a7c6dc734d119df05a8dd4ff5074'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.403205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e071d4113c214f10b34faea61389f867'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.403850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e071d4113c214f10b34faea61389f867'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.404565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e071d4113c214f10b34faea61389f867'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.417967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e071d4113c214f10b34faea61389f867'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.424372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1060/2147] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:55 2022] DELETE /identity/v3/users/b4f5a7c6dc734d119df05a8dd4ff5074 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:55.460276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1088/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:55 2022] DELETE /identity/v3/users/e071d4113c214f10b34faea61389f867 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:55.676818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:55.678320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:55.734779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:55.775512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563266cf-5a85-4c04-9335-7a360b1b2727 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.836898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.837532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.837838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.838108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] PATH_INFO: `/v3/projects/57bef94217e044039a21e2e41f5081de` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.839730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bef94217e044039a21e2e41f5081de)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.841663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.881625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:55.889441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:55.898092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff13fe1-3a2c-4ee6-8d80-7cc2c76840cc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:55.907145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:55.908260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.908648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.909046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] PATH_INFO: `/v3/projects/a185a7668c814f8eb1e56613da31118b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:55.910354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a185a7668c814f8eb1e56613da31118b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:55.913688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:55.954163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bef94217e044039a21e2e41f5081de', 'request_id': 'req-919ac323-8c73-4f82-83e2-4b868b8d6862'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.962880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bef94217e044039a21e2e41f5081de', 'request_id': 'req-919ac323-8c73-4f82-83e2-4b868b8d6862'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.973391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bef94217e044039a21e2e41f5081de', 'request_id': 'req-919ac323-8c73-4f82-83e2-4b868b8d6862'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.978977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a185a7668c814f8eb1e56613da31118b', 'request_id': 'req-40606b32-3b15-454c-956a-5132f920e100'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.986468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a185a7668c814f8eb1e56613da31118b', 'request_id': 'req-40606b32-3b15-454c-956a-5132f920e100'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.987034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] The token cache is being invalidate because project 57bef94217e044039a21e2e41f5081de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:55.987330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.988106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-919ac323-8c73-4f82-83e2-4b868b8d6862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:55.989685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:55.990797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1061/2149] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:55 2022] DELETE /identity/v3/projects/57bef94217e044039a21e2e41f5081de => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:55.995518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:55.996072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:55.996349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.000500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a185a7668c814f8eb1e56613da31118b', 'request_id': 'req-40606b32-3b15-454c-956a-5132f920e100'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.018944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] The token cache is being invalidate because project a185a7668c814f8eb1e56613da31118b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:56.019356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.020098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-40606b32-3b15-454c-956a-5132f920e100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.021893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1089/2150] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:55 2022] DELETE /identity/v3/projects/a185a7668c814f8eb1e56613da31118b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:56.026370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.026370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.028726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a593-173d-4093-8222-f643ca1d9e58 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.029083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a593-173d-4093-8222-f643ca1d9e58 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.029466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a593-173d-4093-8222-f643ca1d9e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.053330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2c26a593-173d-4093-8222-f643ca1d9e58 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.055411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2c26a593-173d-4093-8222-f643ca1d9e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.122336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1062/2151] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.123421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.124946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.138153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b38ff-1186-4c9c-9ead-186f45c39bfb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.143365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.144021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.144286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.144559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] PATH_INFO: `/v3/projects/bf371b25552144a1b91eba389c49b992` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.145845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf371b25552144a1b91eba389c49b992)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.147770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.161298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:56.174624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1090/2152] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.174624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d509aa-6b67-4f11-9b95-d276313fb03d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.174624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d509aa-6b67-4f11-9b95-d276313fb03d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.174624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d509aa-6b67-4f11-9b95-d276313fb03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.191570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-73d509aa-6b67-4f11-9b95-d276313fb03d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.192783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-73d509aa-6b67-4f11-9b95-d276313fb03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.200873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf371b25552144a1b91eba389c49b992', 'request_id': 'req-3c890d3b-15ed-47a8-98fc-401b887558ee'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.207287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf371b25552144a1b91eba389c49b992', 'request_id': 'req-3c890d3b-15ed-47a8-98fc-401b887558ee'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.215643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf371b25552144a1b91eba389c49b992', 'request_id': 'req-3c890d3b-15ed-47a8-98fc-401b887558ee'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.226435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] The token cache is being invalidate because project bf371b25552144a1b91eba389c49b992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:56.226723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.227356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-3c890d3b-15ed-47a8-98fc-401b887558ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.228901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1063/2153] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:56 2022] DELETE /identity/v3/projects/bf371b25552144a1b91eba389c49b992 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:56.230789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.231070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.231338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.251939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.253180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.259851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1091/2154] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.261206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.261490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.261814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.265621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1064/2155] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.266428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.278942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c663b-92fb-44cb-b4f1-0c81d40f4677 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.283553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.284143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.284419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.284687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.284786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.285327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:56.285624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.285991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.287255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b9893-66d6-4e19-b946-f3090f5c787b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.295526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1092/2156] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.296339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.300076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1065/2157] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:56.304373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.304658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.304927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.306946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccacc1-ae52-4da6-9cdc-af5d3bff8e87 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.311543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.312285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.312413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.312654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.313293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:56.313590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.316031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.325634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.327316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.328650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1093/2158] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:56.329244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.337516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1066/2159] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.338652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.343871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292423d-d6c2-4070-b59c-425bd34d7506 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.348448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.349054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.349330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.349596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.350267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.350547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23421b10-f6a7-499d-aa75-9f3f48048fb0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.352413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.356195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.356792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.357071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.357348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.358032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.361577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.385974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1094/2160] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.385974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.403889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1067/2161] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.403889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.437066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:56.454341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078dabd-ef4b-44fc-965e-9a239a178817 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.466676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce77f52-b3a7-4086-8674-f9a6ac3b3a9e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.471282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.471885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.472163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.472431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.473091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.475033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.499437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1068/2162] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.499437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.501014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.512961 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.513342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ecccd-4300-4899-9120-22a2108a8335 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.513505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.513802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.514074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.514718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:56.515017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.516631 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.518407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.519052 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.519333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.519630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] PATH_INFO: `/v3/users/dce7b291f44b4b33b1727d5da994a2b8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.520773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce7b291f44b4b33b1727d5da994a2b8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.522768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.531805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1095/2163] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:56.531805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.538870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885561c-973a-4803-bc61-b3c88640b034 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.544076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.544650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.544927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.545190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.545984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.547757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.564367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:56.569772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1096/2164] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/users => generated 293 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.571051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.576874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce7b291f44b4b33b1727d5da994a2b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.581054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4614ddb-d889-4a06-95ce-61809841b9df admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.584408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce7b291f44b4b33b1727d5da994a2b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.585161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce7b291f44b4b33b1727d5da994a2b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.585942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce7b291f44b4b33b1727d5da994a2b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.586292 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.586875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.587151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.587477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.588054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.590177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.594992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce7b291f44b4b33b1727d5da994a2b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:56.600848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1097/2165] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:56.601700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.603715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.613801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1069/2166] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 07:59:56 2022] DELETE /identity/v3/users/dce7b291f44b4b33b1727d5da994a2b8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:56.614627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.615944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.617101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e7ccd-af03-4b92-b050-526354295038 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.677153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.677785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.678064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.678332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.678957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.680836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f14a5-0ff3-4000-a66a-595d5d0dadc8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.686161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65779a3-1a2f-4b2c-bdc6-55d47c62bb8e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.690846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.691453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.691896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.692061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.692727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.694416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.705726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1070/2167] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:56.706637 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.708282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.711745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1098/2168] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/projects => generated 397 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.711745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.711745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.711745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.723783 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d64ac5-25ba-4be3-91ef-d172c249350a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.730265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:56.731489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.783322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:56.799651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.802707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.802989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.803262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.804158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.805885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.807598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1099/2169] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:56.811145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.821912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1071/2170] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:56.822769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.832300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ff717-c220-4c04-9a44-4ec9f2b3b1ab None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.838035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06dbb-9b2e-4dc0-b64a-7a364c602a3b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.838732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.840433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.840810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.841223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.842193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.843610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.844287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.844415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.845373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.845903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.846650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.848735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.860416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1100/2171] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:56.860416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.862977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:56.871725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1072/2172] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:56 2022] POST /identity/v3/users => generated 299 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:56.872346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.887681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0ec96-bb37-44b6-b0a5-09c7366392bf admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.894352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:56.900813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:56.900813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:56.900813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] PATH_INFO: `/v3/projects/9c2591ffe2f0453c9166b8170de4d369/users/5e72add6c34340caaea5595dd89f8668/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:56.917678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2591ffe2f0453c9166b8170de4d369, user_id=5e72add6c34340caaea5595dd89f8668, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:56.920454 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:56.932899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645edc16-9607-4f04-b51a-13300d4f0505 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:56.936088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1073/2173] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:56 2022] PUT /identity/v3/projects/9c2591ffe2f0453c9166b8170de4d369/users/5e72add6c34340caaea5595dd89f8668/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:56.936924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:56.942732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:57.003282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.003910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.004196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.004469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.018417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159e313-d7a4-4056-b78d-dbec82079a72 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.024610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.026210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.026644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.027648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] PATH_INFO: `/v3/projects/b6c9763de3024b1891ec652d653587db/users/471db9c0db6544a0a7661f2c74eff300/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.047326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c9763de3024b1891ec652d653587db, user_id=471db9c0db6544a0a7661f2c74eff300, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.050416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.069395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1074/2174] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:56 2022] PUT /identity/v3/projects/b6c9763de3024b1891ec652d653587db/users/471db9c0db6544a0a7661f2c74eff300/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:57.070557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.085490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.087277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef602555-efa6-4a4a-8468-4ba4b5626e76 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.098561 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1101/2175] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:57.100023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.100368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.100713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.137710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] MFA Rules not processed for user `5e72add6c34340caaea5595dd89f8668`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:57.139137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.157023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f850be4-5da6-49ec-a7e3-86cb3623c823 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.164538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.166109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.166632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.167205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.168489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.172527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46338139-b0f7-4ca1-bc7d-e8ea9a18ff52 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.182356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:57.192798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1075/2176] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:57.196699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.197453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.197960 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.239905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] MFA Rules not processed for user `471db9c0db6544a0a7661f2c74eff300`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:57.241549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.258046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1102/2177] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:57.259002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.272534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9c191-0075-4615-88a9-c21cb660b915 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.278226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.278989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.279358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.279684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.280410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.282286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.298322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1103/2178] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:57.298322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.309407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d37104-bced-40b8-b7b0-93d8b4400e34 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.344533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.345485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.345588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.345891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.346544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.348540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.362679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1076/2179] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:57.363636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.380647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ebb6d-242c-4fcd-a556-d4523d10d715 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.389125 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1104/2180] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:57.389998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.393380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.396700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.398203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.398498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.398776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] PATH_INFO: `/v3/projects/2ea1aa042af346adaec8f9c6e6d4be97/users/450fd77a118d4aeb91d9302982696229/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.413165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:57.423411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea1aa042af346adaec8f9c6e6d4be97, user_id=450fd77a118d4aeb91d9302982696229, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.427363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.443817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1077/2181] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:57 2022] PUT /identity/v3/projects/2ea1aa042af346adaec8f9c6e6d4be97/users/450fd77a118d4aeb91d9302982696229/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:57.453828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.488261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877cfe-866a-4d18-b0f2-e0d5e56c8a2a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.527785 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3729b-ab58-4a93-97c7-de252c2f2126 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.534152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.534887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.535270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.536117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.536287 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.538903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf357ea8-86b4-4a2e-aa3e-76b2b1a1e7c7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.552161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.552766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.553089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.553343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.563468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1078/2182] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:57.566823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.567189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.567524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.599666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] MFA Rules not processed for user `450fd77a118d4aeb91d9302982696229`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:57.600887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.623117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.624742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.637352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1105/2183] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:57.638081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.649134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:57.653167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59390f4-52a3-44ba-a631-3cc591d1f444 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.667164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.668471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.668752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.669135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.669952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.671845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.679945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1079/2184] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:57.688822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1106/2185] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:57.693247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.698975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.701878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.710277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce48e2-84d0-43e4-a4da-f0080e3ed72b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.716176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.716342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90884-504c-4e83-bbdf-79e6d2fe92b9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.716691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.717020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.717353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.718703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.721038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.734927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1080/2186] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:57.738288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.742261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.742930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.743203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.744179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.754306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5c8ce-fecb-457d-9512-48e9199238b4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.761559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.762049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.762350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.763033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] PATH_INFO: `/v3/projects/54607fa413364f70ad81dfa51afdb849/users/377cce2d814f49cf8b9e73ba508a1c07/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.768760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.770620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.784107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1107/2187] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:57.785236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:57.786127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=54607fa413364f70ad81dfa51afdb849, user_id=377cce2d814f49cf8b9e73ba508a1c07, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.787173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.789355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788f05b-dbb3-45ff-b073-18562a6880bd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:57.817329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1081/2188] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:57 2022] PUT /identity/v3/projects/54607fa413364f70ad81dfa51afdb849/users/377cce2d814f49cf8b9e73ba508a1c07/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:57.817992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9341b34-9350-4b2c-baf6-5528b3113103 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:57.819698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.819971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.820240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.865756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] MFA Rules not processed for user `377cce2d814f49cf8b9e73ba508a1c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:57.870046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:57.945474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:57.965289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:57.967460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:57.975315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:57.975800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] PATH_INFO: `/v3/projects/743315b0ded543eb8848996e6f720a17` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:57.978994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=743315b0ded543eb8848996e6f720a17)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:57.981000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.062311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1082/2189] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:58.062877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.091146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7eb15-204d-4f25-8bd3-84adcff1a902 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.096687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743315b0ded543eb8848996e6f720a17', 'request_id': 'req-285089bb-e153-493f-b2f3-d7da066788bb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:58.105603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743315b0ded543eb8848996e6f720a17', 'request_id': 'req-285089bb-e153-493f-b2f3-d7da066788bb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:58.116784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743315b0ded543eb8848996e6f720a17', 'request_id': 'req-285089bb-e153-493f-b2f3-d7da066788bb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:58.132341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] The token cache is being invalidate because project 743315b0ded543eb8848996e6f720a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:59:58.133919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:58.135526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-285089bb-e153-493f-b2f3-d7da066788bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:59:58.142979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1108/2190] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 07:59:57 2022] DELETE /identity/v3/projects/743315b0ded543eb8848996e6f720a17 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:58.144351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ac33-5cba-4dc9-91d7-a668c52c641c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.144610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ac33-5cba-4dc9-91d7-a668c52c641c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.144890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ac33-5cba-4dc9-91d7-a668c52c641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.169023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c327ac33-5cba-4dc9-91d7-a668c52c641c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:58.170376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c327ac33-5cba-4dc9-91d7-a668c52c641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:58.193962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.196298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.197272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.198363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.199660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.202230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.241535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1083/2191] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:58.246379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.250538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:58.260859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1109/2192] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:59:58.270777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.271154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.271911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.311251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:58.312869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:58.319973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:58.347030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6d676-3eea-4feb-8745-f41aca7f64bc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.351977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.352740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.353024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.353295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.353963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.356426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.383775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1084/2193] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:59:58.387478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.399421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bd19d-2c53-4824-a23b-f95974f21d3b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.409497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.411312 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.411938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.412263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.412943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.415396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.424080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1110/2194] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:58.428452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.444727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1085/2195] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:58.451873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.459729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2e41-8f1a-4133-9d37-741600cb6a4a None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.469157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56740ed3-81bc-48a7-99b9-7b9570eda547 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.478114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.480535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.481031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.481632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] PATH_INFO: `/v3/projects/e2189cdd738142ccbc26768039187eec/users/d152ea1c700c4b05a5491f817ab5713a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.482408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.483036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.483322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.483586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.484244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:59:58.484545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.492495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.516022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2189cdd738142ccbc26768039187eec, user_id=d152ea1c700c4b05a5491f817ab5713a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.520320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1111/2196] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:59:58.523821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.524497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.545953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083185d-e46d-4079-9314-f88ea8b2bfd1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.549184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:58.555480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1086/2197] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:58 2022] PUT /identity/v3/projects/e2189cdd738142ccbc26768039187eec/users/d152ea1c700c4b05a5491f817ab5713a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:58.559347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.675548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.676840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.677127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.677404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.678089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.680071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.687371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf86049-a0b1-40d9-a598-de4475aa86b8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.693870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.695023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.695607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.695877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.696813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.698585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.710975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1087/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:58.714920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1112/2199] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:58.715773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.715953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.726051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53800dd4-c106-41a0-8fce-370d700ab422 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.780439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4013c5-14af-4d9d-a66a-62c562de010c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.785086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.785272 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.785774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.785936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.786056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.786213 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.786330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] PATH_INFO: `/v3/projects/e2189cdd738142ccbc26768039187eec/users/d152ea1c700c4b05a5491f817ab5713a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.786484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.787172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.788228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2189cdd738142ccbc26768039187eec, user_id=d152ea1c700c4b05a5491f817ab5713a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.789120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.790947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee05fa-906c-4106-bd1c-8d7a6551770b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.802431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1088/2200] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:58 2022] PUT /identity/v3/projects/e2189cdd738142ccbc26768039187eec/users/d152ea1c700c4b05a5491f817ab5713a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:59:58.813508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.813818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.814093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.816909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1113/2201] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:59:58.828257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.841486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a338c7-9cbb-4743-8478-f7d1f8e8d599 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.845468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] MFA Rules not processed for user `d152ea1c700c4b05a5491f817ab5713a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:58.847250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:58.851216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:58.912951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.915578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.916414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.917428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.918960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.922373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.935196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1089/2202] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:58.938685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1114/2203] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:59:58.941847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.947159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.949796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:58.953051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd59f4-82d0-446e-ab2a-717c4051041a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.959238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.959979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.960288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.960560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:58.961214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:58.962859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:58.974605 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1090/2204] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:59:58.979064 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:58.985477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:59:58.989867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fa837-b867-4041-b967-2f9e0e2ad771 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:58.995766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:58.996418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:58.996707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:58.996963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] PATH_INFO: `/v3/projects/44a6ed00e5ad40be9c50d72808559c8e/users/eb1aff5c8217437ea3436ccf9a892054/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:59.011663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a6ed00e5ad40be9c50d72808559c8e, user_id=eb1aff5c8217437ea3436ccf9a892054, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:59.014379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b421f59-a2f5-4b92-8ff1-59c311fcf0d0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:59.016846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bc021-2cfd-433b-a006-03a92d733106 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:59.021233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:59.022002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:59.022288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:59.022559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:59.030062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1091/2205] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 07:59:58 2022] PUT /identity/v3/projects/44a6ed00e5ad40be9c50d72808559c8e/users/eb1aff5c8217437ea3436ccf9a892054/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:59:59.034938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:59.035222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:59.035493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:59.036101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:59.037531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:59.047312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1115/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:59:59.053716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:59:59.054941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:59.119414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1092/2207] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 07:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:59:59.131275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:59.132780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:59.193431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62443f51-0c46-4e52-a635-0d788954602d service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:59.251162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:59.251777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:59.252062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:59.252336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:59.262748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:59:59.264737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28acee4-d33b-42c3-935e-e6f5fb2d4eb9 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:59:59.277060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1116/2208] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:59:59.866743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:59:59.870745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:59:59.945428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae00f1-2e24-405f-be51-cb5483506107 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:59:59.951063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:59:59.951936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:59:59.952308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:59:59.952678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:59:59.954920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:00.062571 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:00.064056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:00.075025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1093/2209] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 07:59:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:00.227343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa24-dc77-477a-977e-22e6e79b0edf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:00.227926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa24-dc77-477a-977e-22e6e79b0edf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:00.227926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5fa24-dc77-477a-977e-22e6e79b0edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:00.257449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:00.267636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d3e5fa24-dc77-477a-977e-22e6e79b0edf None None] MFA Rules not processed for user `eb1aff5c8217437ea3436ccf9a892054`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:00.269116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d3e5fa24-dc77-477a-977e-22e6e79b0edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:00.348934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1117/2210] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:00.358451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:00.368449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b71bc9-59c2-4331-aab9-905f829937fc service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:00.372953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:00.373976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:00.374502 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:00.374869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:00.388256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:00.390017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:00.401251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1094/2211] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:07.039285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cb17-79bc-4aa9-ae74-883d9f762427 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:07.039832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cb17-79bc-4aa9-ae74-883d9f762427 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:07.039832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cb17-79bc-4aa9-ae74-883d9f762427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:07.060200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-08d4cb17-79bc-4aa9-ae74-883d9f762427 None None] MFA Rules not processed for user `eb1aff5c8217437ea3436ccf9a892054`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:07.061396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-08d4cb17-79bc-4aa9-ae74-883d9f762427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:07.068489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1118/2212] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:07.611334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:07.612825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:07.629943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b549-646e-459a-92f5-263545c5e4d6 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:07.648463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:07.649023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:07.649404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:07.649902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] PATH_INFO: `/v3/users/471db9c0db6544a0a7661f2c74eff300` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:07.652256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471db9c0db6544a0a7661f2c74eff300)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:07.654297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:07.679145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:07.718818 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471db9c0db6544a0a7661f2c74eff300'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:07.732066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471db9c0db6544a0a7661f2c74eff300'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:07.733757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471db9c0db6544a0a7661f2c74eff300'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:07.734535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471db9c0db6544a0a7661f2c74eff300'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:07.752135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471db9c0db6544a0a7661f2c74eff300'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:07.773291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1095/2213] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:07 2022] DELETE /identity/v3/users/471db9c0db6544a0a7661f2c74eff300 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:07.778057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:07.778338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:07.778606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:07.799433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:07.800645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:07.873093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1119/2214] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:07.883992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:07.888481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:07.958567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe13a6-4765-4d7a-baed-574f76567802 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:07.964407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:07.965320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:07.965668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:07.966039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:07.977100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:07.978579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556b8b0-98cc-4116-bf95-8cd980fa532e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:07.990330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1096/2215] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:08.531450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:08.533264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:08.557343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8226a1-17d7-40a0-a264-e40285869003 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:08.570027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:08.571732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:08.572085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:08.572424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] PATH_INFO: `/v3/users/450fd77a118d4aeb91d9302982696229` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:08.574098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=450fd77a118d4aeb91d9302982696229)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:08.576162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:08.632088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450fd77a118d4aeb91d9302982696229'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.640390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450fd77a118d4aeb91d9302982696229'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.641136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450fd77a118d4aeb91d9302982696229'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.641764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450fd77a118d4aeb91d9302982696229'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.650377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450fd77a118d4aeb91d9302982696229'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.669114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1120/2216] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:08 2022] DELETE /identity/v3/users/450fd77a118d4aeb91d9302982696229 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:08.674142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:08.675350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:08.675640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:08.695718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:08.697172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:08.758775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1097/2217] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:08.767444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:08.771113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:08.781476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:08.784685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:08.800599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228899c4-1110-45b3-b95c-78d1ae8e7bb1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:08.806026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:08.806825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:08.807104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:08.807427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] PATH_INFO: `/v3/projects/b6c9763de3024b1891ec652d653587db` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:08.809460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c9763de3024b1891ec652d653587db)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:08.814912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:08.816899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:08.903609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:08.928119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae288d-f579-4dab-90ee-7b111f0fd15c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:08.932937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c9763de3024b1891ec652d653587db', 'request_id': 'req-adaec24a-337b-47e8-940b-abfd8043016e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.953909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c9763de3024b1891ec652d653587db', 'request_id': 'req-adaec24a-337b-47e8-940b-abfd8043016e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.967586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c9763de3024b1891ec652d653587db', 'request_id': 'req-adaec24a-337b-47e8-940b-abfd8043016e'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.994164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] The token cache is being invalidate because project b6c9763de3024b1891ec652d653587db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:08.994478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:08.996978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:08.997690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:08.998870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:08.998870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:08.999217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-adaec24a-337b-47e8-940b-abfd8043016e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:09.006727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1121/2218] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:08 2022] DELETE /identity/v3/projects/b6c9763de3024b1891ec652d653587db => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:09.008889 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.009195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.017907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.044740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:09.046274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:09.128975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.130473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899f3a5-a939-455f-8deb-059114940b24 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.142841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1098/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:09.143058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1122/2219] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:09.144068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.144356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.144629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.167573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:09.168811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:09.183080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1099/2221] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:09.184562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.198701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d28a0-303a-4635-ac29-d8a34e7113e7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.211678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.212605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.213364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.214783 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.217156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:09.217622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.220750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.246112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1123/2222] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:09.248296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.274070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf178bd5-cd3d-4ad5-b7b4-d70d37b54662 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.280159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.280809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.281093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.281426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.282309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.284341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.324580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1100/2223] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:09 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:09.326942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.338175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af96c4-29f3-40ee-91d0-28be43b0fbeb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.413662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.414488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.414902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.415267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.416191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.420515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.466595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1124/2224] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:09 2022] POST /identity/v3/users => generated 285 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:09.471725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.487076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738547d7-0759-4ac6-becb-14fbbb7a97e6 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.500234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.500901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.501186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.501465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.503772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.505516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.531590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1101/2225] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:09.533413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.554276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901aafb-74ee-4ea6-ba1b-67c9e793217a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.561134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.561687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.562008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.562276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.562959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.564557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e056bfa-88fc-43fc-8d80-42c298e7c8c6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.566247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:09.589911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1125/2226] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:09.602665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.619896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8541a-5071-410f-ab26-595b28d532cd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.635044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.638042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.638580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.639807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] PATH_INFO: `/v3/projects/d5e85beab3174ee9bfa72c4f6dcebccc/users/5b1aaa91cad34cdc9fe908f07d40b123/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.662861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e85beab3174ee9bfa72c4f6dcebccc, user_id=5b1aaa91cad34cdc9fe908f07d40b123, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:09.667475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:09.702286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1102/2227] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:09 2022] PUT /identity/v3/projects/d5e85beab3174ee9bfa72c4f6dcebccc/users/5b1aaa91cad34cdc9fe908f07d40b123/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:09.704444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.704728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.705763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.757964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] MFA Rules not processed for user `5b1aaa91cad34cdc9fe908f07d40b123`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:09.757964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:09.772777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.776847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:09.834897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1126/2228] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:09.842485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:09.850726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:09.852956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:09.862408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7c413-c8c3-44ce-b0f3-6d1b6f974a6d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:09.866684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:09.867306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:09.867582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:09.878768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:09.881086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:10.007993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0638c9-f327-4a67-bd09-04a5ff3d81b8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:10.013886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:10.014634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.015021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.015401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.022757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:10.025491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:10.029324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:10.033390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e9f02-5ed1-4701-80e1-d948c934d85e service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:10.044225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1103/2229] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:00:10.056083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1127/2230] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:00:10.164074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:10.166003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:10.334405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa3138-9317-4943-ae03-9c3c7754b947 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:10.349388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:10.350449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.350831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.351210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] PATH_INFO: `/v3/projects/2ea1aa042af346adaec8f9c6e6d4be97` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.354734 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea1aa042af346adaec8f9c6e6d4be97)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:10.360797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:10.449473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea1aa042af346adaec8f9c6e6d4be97', 'request_id': 'req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:10.461887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea1aa042af346adaec8f9c6e6d4be97', 'request_id': 'req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:10.478989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea1aa042af346adaec8f9c6e6d4be97', 'request_id': 'req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:10.497844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] The token cache is being invalidate because project 2ea1aa042af346adaec8f9c6e6d4be97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:10.498243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:10.499240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ec84e7d-ca4b-41fb-aeb7-148f0c8cb3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:10.501190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1104/2231] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:10 2022] DELETE /identity/v3/projects/2ea1aa042af346adaec8f9c6e6d4be97 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:10.506797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.507094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.507376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.550768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:10.551939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:10.709588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1128/2232] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:10.712243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.712699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.712871 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.770289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:10.771477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:10.793059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1105/2233] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:10.797924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:10.825329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717bf23-0747-4885-8151-66c01b9d9a1e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:10.837028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:10.838429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.839454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.839899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.841135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:10.841368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:10.843965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:10.849373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:10.868091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1129/2234] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:10.868948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:10.891574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224cbf-039a-4e1b-888e-a21da3fa50c9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:10.911997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:10.912830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:10.912964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:10.913165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:10.913861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:10.915822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:10.947459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:10.971935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1106/2235] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:10 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:10.976098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:10.990313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08c6ae-16ea-424a-a992-1e0ba6ce5c13 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:11.080340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:11.083086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.083443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.083790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.084511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:11.086817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:11.133789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1130/2236] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:10 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:11.138132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:11.154628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb420229-c832-43a1-b5be-14adbbbdce9d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:11.184977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:11.186104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.192136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.192495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.193194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:11.194865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:11.250883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:11.263648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1107/2237] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:11.266054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5fe57-ef7c-415d-b9dc-154c05e26f25 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:11.294296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:11.294740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.295074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.295513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.296080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:11.297724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fcc6e-93e4-4939-9f37-dc436799b9ce admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:11.316167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1131/2238] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:11.321148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:11.348198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c109b5-91af-4a18-813e-eb3b9b41afd8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:11.359850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:11.362059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.375190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.375190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] PATH_INFO: `/v3/projects/29cd3d666355422e8ac5802fd07de461/users/67ec6b7d73494d9bb0294c21277407ce/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.384267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cd3d666355422e8ac5802fd07de461, user_id=67ec6b7d73494d9bb0294c21277407ce, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:11.388281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:11.409304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1108/2239] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:11 2022] PUT /identity/v3/projects/29cd3d666355422e8ac5802fd07de461/users/67ec6b7d73494d9bb0294c21277407ce/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:00:11.414949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0104-e52b-4c5c-8680-6801d6609a9f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.415233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0104-e52b-4c5c-8680-6801d6609a9f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.416217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc0104-e52b-4c5c-8680-6801d6609a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.452450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5ccc0104-e52b-4c5c-8680-6801d6609a9f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:11.453863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5ccc0104-e52b-4c5c-8680-6801d6609a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:11.650969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1132/2240] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:11.664173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:11.666210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:11.756470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11953b-61a0-4045-ad45-2648efa6f938 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:11.762266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:11.763148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:11.763436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:11.763709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:11.775543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:11.777777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e972b-2d66-4540-b18b-2c1a5c6dcc79 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:11.789523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1109/2241] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:12.395774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:12.396096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:12.396581 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:12.398154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1133/2242] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:00:12.419328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:12.420612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:12.420956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:12.445369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:12.447480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:12.528712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:12.530577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:12.565311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70ccc6-55b4-434b-9eb5-19cee98e8aa3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:12.578383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:12.579079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:12.579781 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:12.580852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] PATH_INFO: `/v3/users/0d7a4ca3d0904c1f8e3b9622fea8af8c` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:12.583349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7a4ca3d0904c1f8e3b9622fea8af8c)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:12.584577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1110/2243] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:00:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:12.587017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:12.651839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:12.654392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:12.711434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa25532-750c-4c2a-b2fb-536fac2de46b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:12.726395 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:12.727474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:12.727945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:12.728914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:12.743881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7a4ca3d0904c1f8e3b9622fea8af8c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:12.747726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:12.749433 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:12.766244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7a4ca3d0904c1f8e3b9622fea8af8c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:12.767079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7a4ca3d0904c1f8e3b9622fea8af8c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:12.768528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7a4ca3d0904c1f8e3b9622fea8af8c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:12.780977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7a4ca3d0904c1f8e3b9622fea8af8c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:12.781538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1111/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:12.788373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:12.809070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1134/2245] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:12 2022] DELETE /identity/v3/users/0d7a4ca3d0904c1f8e3b9622fea8af8c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:13.378944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:13.380459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.396528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8c39a-76de-499b-a183-319c91d93748 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:13.455583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:13.481422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:13.482666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.483715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.484234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] PATH_INFO: `/v3/users/5e72add6c34340caaea5595dd89f8668` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.485837 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e72add6c34340caaea5595dd89f8668)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:13.487915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:13.512286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:13.513751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.535326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83a156-eac8-4abe-b73d-e3e95d6c526f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:13.540428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:13.541274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.541492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.541902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] PATH_INFO: `/v3/projects/074dbd6970034af1b02996d318d42e88` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.543153 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=074dbd6970034af1b02996d318d42e88)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:13.545128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:13.555853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e72add6c34340caaea5595dd89f8668'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.567019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e72add6c34340caaea5595dd89f8668'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.567908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e72add6c34340caaea5595dd89f8668'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.569368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e72add6c34340caaea5595dd89f8668'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.582277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7d431886-fe48-47c0-a9ea-a43f828e555b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e72add6c34340caaea5595dd89f8668'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.623978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1112/2246] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:13 2022] DELETE /identity/v3/users/5e72add6c34340caaea5595dd89f8668 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:13.626290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074dbd6970034af1b02996d318d42e88', 'request_id': 'req-80da5da9-abad-4390-9c14-aca8da3ca513'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.629716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.630059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.630302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.636517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074dbd6970034af1b02996d318d42e88', 'request_id': 'req-80da5da9-abad-4390-9c14-aca8da3ca513'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.646328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074dbd6970034af1b02996d318d42e88', 'request_id': 'req-80da5da9-abad-4390-9c14-aca8da3ca513'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.654510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:13.655786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.663296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] The token cache is being invalidate because project 074dbd6970034af1b02996d318d42e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:13.663716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.664647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80da5da9-abad-4390-9c14-aca8da3ca513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:13.670136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1135/2247] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:13 2022] DELETE /identity/v3/projects/074dbd6970034af1b02996d318d42e88 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:13.673607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493230d0-b12e-4b68-a412-b94fe3c0b20b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.673956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493230d0-b12e-4b68-a412-b94fe3c0b20b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.674275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493230d0-b12e-4b68-a412-b94fe3c0b20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.709880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-493230d0-b12e-4b68-a412-b94fe3c0b20b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:13.711559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-493230d0-b12e-4b68-a412-b94fe3c0b20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.752256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1113/2248] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:13.770165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:13.771621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.806115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1136/2249] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:13.811726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.816298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.816601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.839230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:13.840635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:13.846935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04221c1-458b-4bde-98dc-3329c1223924 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:13.850658 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1137/2250] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:13.854671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:13.855207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:13.855453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.855805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.856148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.866951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:13.867632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c7810-dd41-49f0-adc5-b2d9f6cba6ac None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:13.868563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:13.873363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:13.874155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.874365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.874636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.875369 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:13.875736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:13.877417 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:13.879782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1114/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:13.891825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1138/2252] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 08:00:13.895931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:13.907434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6316b-a301-462e-9978-06f81a86d8ea service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:13.912587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:13.913675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:13.913675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:13.914167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:13.914904 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:13.917688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:13.925123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:13.981221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1115/2253] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:13 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:13.982644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.006447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5e97-9d9f-42be-8ceb-42a53e56d1a3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.111363 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:14.131667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.134330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.134615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.134890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.135643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:14.138075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:14.193304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1139/2254] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:13 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:14.194563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.212744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e8691-1439-46d7-97ef-0d6787dd10db admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.228270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.230826 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.231475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.233371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.235033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:14.240858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:14.265059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1116/2255] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:14.275854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.288806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ec10-4a05-44f8-827c-7332950cca09 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.295224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.295821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.296103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.296425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.297075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:14.298769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d090d29-c014-4d44-b8b0-bf2362c17bdc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:14.311050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1140/2256] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:14.316225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.338974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572665c2-4cf3-4062-9983-96d0b7dc8751 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.347239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.348830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.349542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.354842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] PATH_INFO: `/v3/projects/189468d0e3034caaa498442603da0609/users/6a3c9d9bf3c8455aa93ad43b10bfd956/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.394679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189468d0e3034caaa498442603da0609, user_id=6a3c9d9bf3c8455aa93ad43b10bfd956, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:14.397549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:14.414768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1117/2257] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:14 2022] PUT /identity/v3/projects/189468d0e3034caaa498442603da0609/users/6a3c9d9bf3c8455aa93ad43b10bfd956/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:14.420092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.420382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.420659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.441469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:14.442725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:14.581851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1141/2258] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:14.606975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.609773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:14.627211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.629511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dac69e-ba82-4974-95e1-c0942882e075 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.630858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:14.637278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.638055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.638535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.638884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] PATH_INFO: `/v3/projects/9c2591ffe2f0453c9166b8170de4d369` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.640405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2591ffe2f0453c9166b8170de4d369)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:14.642620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:14.786056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c94b-d28e-45f0-ac04-64ab0f8627f8 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.791234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2591ffe2f0453c9166b8170de4d369', 'request_id': 'req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:14.805800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2591ffe2f0453c9166b8170de4d369', 'request_id': 'req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:14.818875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2591ffe2f0453c9166b8170de4d369', 'request_id': 'req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:14.837322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] The token cache is being invalidate because project 9c2591ffe2f0453c9166b8170de4d369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:14.837646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:14.846507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:14.850226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1118/2259] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:14 2022] DELETE /identity/v3/projects/9c2591ffe2f0453c9166b8170de4d369 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:14.855139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:14.856650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:14.879368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:14.879958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:14.880194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:14.880716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:14.885021 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:14.992368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d0506-67e9-4d56-bd1c-5ec4ea7f9216 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:14.999567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:15.000532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:15.000916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:15.001098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] PATH_INFO: `/v3/users/377cce2d814f49cf8b9e73ba508a1c07` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:15.002512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377cce2d814f49cf8b9e73ba508a1c07)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:15.004863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:15.015529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:15.033584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:15.035329 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:15.047590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1142/2260] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:00:15.086231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:15.099686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377cce2d814f49cf8b9e73ba508a1c07'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.111431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377cce2d814f49cf8b9e73ba508a1c07'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.112171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377cce2d814f49cf8b9e73ba508a1c07'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.113005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377cce2d814f49cf8b9e73ba508a1c07'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.123116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377cce2d814f49cf8b9e73ba508a1c07'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.144625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1119/2261] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:14 2022] DELETE /identity/v3/users/377cce2d814f49cf8b9e73ba508a1c07 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:15.749200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:15.764775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54908d7-6aa8-4974-9003-3d3a8efdf994 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:15.874229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:15.875484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:15.876032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:15.876946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] PATH_INFO: `/v3/projects/54607fa413364f70ad81dfa51afdb849` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:15.879720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54607fa413364f70ad81dfa51afdb849)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:15.883317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:15.976918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54607fa413364f70ad81dfa51afdb849', 'request_id': 'req-58cc6bd6-6df2-482c-8de8-10b5619ab485'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.987011 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54607fa413364f70ad81dfa51afdb849', 'request_id': 'req-58cc6bd6-6df2-482c-8de8-10b5619ab485'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:15.999469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54607fa413364f70ad81dfa51afdb849', 'request_id': 'req-58cc6bd6-6df2-482c-8de8-10b5619ab485'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.022200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] The token cache is being invalidate because project 54607fa413364f70ad81dfa51afdb849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:16.023514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.024444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.028922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1143/2262] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:15 2022] DELETE /identity/v3/projects/54607fa413364f70ad81dfa51afdb849 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:16.031426 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:16.033682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:16.069241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20768b30-d179-446f-a341-23f2be8a6595 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:16.157626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:16.158306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:16.158687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:16.159068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] PATH_INFO: `/v3/users/d152ea1c700c4b05a5491f817ab5713a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:16.160337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d152ea1c700c4b05a5491f817ab5713a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:16.162702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:16.245831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd152ea1c700c4b05a5491f817ab5713a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.257903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd152ea1c700c4b05a5491f817ab5713a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.258914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd152ea1c700c4b05a5491f817ab5713a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.259798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd152ea1c700c4b05a5491f817ab5713a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.274681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-390d796e-8ead-4bfd-a546-b957cd3210e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd152ea1c700c4b05a5491f817ab5713a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:16.313425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1120/2263] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:16 2022] DELETE /identity/v3/users/d152ea1c700c4b05a5491f817ab5713a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:16.976342 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:16.979997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:17.002931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc6bd6-6df2-482c-8de8-10b5619ab485 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.082451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.083104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.083383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.083651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] PATH_INFO: `/v3/projects/e2189cdd738142ccbc26768039187eec` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.084862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2189cdd738142ccbc26768039187eec)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.086990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.097807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:17.201455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2189cdd738142ccbc26768039187eec', 'request_id': 'req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:17.247886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2189cdd738142ccbc26768039187eec', 'request_id': 'req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:17.265195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2189cdd738142ccbc26768039187eec', 'request_id': 'req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:17.298558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] The token cache is being invalidate because project e2189cdd738142ccbc26768039187eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:17.298953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:17.299685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9ce729c7-50d3-46fe-ae28-a5ffee3776a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:17.302044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1144/2264] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:16 2022] DELETE /identity/v3/projects/e2189cdd738142ccbc26768039187eec => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:17.307846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.308388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.308388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.369441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:17.375914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:17.427421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:17.525620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1121/2265] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:17.526239 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.526582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.526857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.586918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:17.588343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:17.610984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:17.611290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1145/2266] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:17.628932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f068-cc39-477a-9022-5ed6607b948c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.637181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.638037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.638806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.639194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.640242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:17.640656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.642940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.661624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1122/2267] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:17.663525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:17.676037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec404a6-bf9b-437f-9e68-a88d91f225d6 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.681921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.683671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.683671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.683671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.684270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.686259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.737574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1146/2268] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:17 2022] POST /identity/v3/projects => generated 391 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:17.737752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:17.751128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a40db-a993-4f97-aac9-f8c3d6e1ace1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.844129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.844944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.845255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.845577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.850682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.853351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.890985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:17.894122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1123/2269] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:17 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:17.908347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb71e4d-d7b8-43cc-8754-e1a1c19f2fd2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.913666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.914354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.914637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.914914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.915571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.917716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.928481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1147/2270] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:17.933484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:17.951829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd21fa-cf11-4074-a754-73b6a768c5c4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:17.959350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:17.960262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:17.960601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:17.960976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:17.961762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:17.964995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931ec62-4291-43b7-9134-8f21b288907e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:17.990409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1124/2271] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:17.994428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.030521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e852a5b-c6cc-4f34-828d-fd941f3bc1b3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.044039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.044945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.045330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.045710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] PATH_INFO: `/v3/projects/acd20feb8ac346f78547e56f06c558e1/users/c1d97e00acbf45e9b75c13422fb7ec1b/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.090994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd20feb8ac346f78547e56f06c558e1, user_id=c1d97e00acbf45e9b75c13422fb7ec1b, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.095470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:18.113423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1148/2272] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:17 2022] PUT /identity/v3/projects/acd20feb8ac346f78547e56f06c558e1/users/c1d97e00acbf45e9b75c13422fb7ec1b/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:18.126075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a07f1b-b643-4d2e-80d9-457ef599fe21 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.126365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a07f1b-b643-4d2e-80d9-457ef599fe21 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.126762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a07f1b-b643-4d2e-80d9-457ef599fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.189519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b9a07f1b-b643-4d2e-80d9-457ef599fe21 None None] MFA Rules not processed for user `c1d97e00acbf45e9b75c13422fb7ec1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:18.191700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b9a07f1b-b643-4d2e-80d9-457ef599fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:18.351396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1125/2273] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:18.357314 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.506303 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a1af-dd71-4bea-a068-bd9c5f6be4ac admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.511760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.512820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.513664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.514185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.530218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.530730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.531026 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.531305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.533209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:18.578390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1149/2274] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/projects => generated 391 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:18.581721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.600136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] MFA Rules not processed for user `67ec6b7d73494d9bb0294c21277407ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:18.604510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:18.638784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:18.647707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:18.723786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1126/2275] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:18.730179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.731894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:18.745469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21897-a75a-464c-b0a9-ae51530e9281 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.748786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999a046-0a2b-427c-a41b-3fc4c4162d3b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.754554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.755297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.755622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.755958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.756318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.756749 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.756994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.757224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.757531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.758160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.759409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:18.760113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:18.802667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1127/2276] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:18.809435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.810690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1150/2277] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/users => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:18.812879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.833649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4f16f-b81e-4cda-a6be-3704f5547284 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.964943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.965900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.966410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.966909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.968258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.969001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19119d-1421-4666-a82c-c43d6b946b09 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:18.970405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:18.982434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:18.983368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:18.983798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:18.984216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:18.984527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1151/2278] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:18.985340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:18.991629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:18.994811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.004744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51d0f-15cb-4e87-b93a-ee5dd4109a04 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.024858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.025605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.026541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.027038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] PATH_INFO: `/v3/projects/4bf66e449a2a47bdbc6369d154149442/users/d53abba75eef430795e046bd4fe07c65/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.061462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1128/2279] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:18 2022] POST /identity/v3/users => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:19.062558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.064462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf66e449a2a47bdbc6369d154149442, user_id=d53abba75eef430795e046bd4fe07c65, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.067406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.094226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1152/2280] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:18 2022] PUT /identity/v3/projects/4bf66e449a2a47bdbc6369d154149442/users/d53abba75eef430795e046bd4fe07c65/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:19.095358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.102725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2c442-45a9-49ab-8977-f8c96ec7e789 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.216080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38daa4-cc8b-43fa-914a-257080cd70fd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.222492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.223628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.223984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.224359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.225090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.226775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.245069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1153/2281] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:19.249468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.250016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.250548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.251386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.252148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.254756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.256925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.266741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467d60-0490-4d33-b0e8-9f5c8efed51e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.281639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.282486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.282771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.283365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1129/2282] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:19.284233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.284758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] PATH_INFO: `/v3/projects/4bf66e449a2a47bdbc6369d154149442/users/d53abba75eef430795e046bd4fe07c65/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.288626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf66e449a2a47bdbc6369d154149442, user_id=d53abba75eef430795e046bd4fe07c65, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.295425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2c37a-7a2d-4cb4-9af9-e17962a3b0fe admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.305409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4d5d-2c6b-4f21-892b-af6eb340641b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.318648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1154/2283] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:19 2022] PUT /identity/v3/projects/4bf66e449a2a47bdbc6369d154149442/users/d53abba75eef430795e046bd4fe07c65/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:19.320216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.321154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.321972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.322471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] PATH_INFO: `/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7/users/33485026d0ed4151abe151d0cdbcd177/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.329946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.330247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.330530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.351627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=071020b20eab4c8e99e1a89e8e4d27f7, user_id=33485026d0ed4151abe151d0cdbcd177, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.354407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:19.355260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.366965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:19.381418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1130/2284] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:19 2022] PUT /identity/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7/users/33485026d0ed4151abe151d0cdbcd177/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:19.385688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.398236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] MFA Rules not processed for user `d53abba75eef430795e046bd4fe07c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:19.400170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:19.476927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32418e-1506-4032-809a-39b940888f2b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.482609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.483232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.483513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.483789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.484447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.486308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.520763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1131/2285] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:19.525027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.568617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855f83-5a12-4ea5-924a-3e9065b435f4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:19.577171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:19.578082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:19.578464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:19.578836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] PATH_INFO: `/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7/users/33485026d0ed4151abe151d0cdbcd177/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:19.583169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071020b20eab4c8e99e1a89e8e4d27f7, user_id=33485026d0ed4151abe151d0cdbcd177, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:19.592578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:19.607574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1155/2286] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:19.628385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1132/2287] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:19 2022] PUT /identity/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7/users/33485026d0ed4151abe151d0cdbcd177/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:19.680520 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:19.683544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:19.850391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abbc16-a415-44ef-951f-6f4caf7bad89 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:20.047257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:20.051911 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:20.052362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:20.055378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:20.244001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:20.250735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:20.261007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5b384-1e1d-4973-9803-c7f2f804b607 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:20.304151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1156/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:20.411044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:20.414841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:20.437046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46773eb-4311-45a0-a2cf-1af50b4afb55 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:20.452981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:20.454811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:20.455396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:20.455684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:20.574016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:20.575587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:20.592455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1133/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:25.087835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:25.088789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:25.089336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:25.131937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] MFA Rules not processed for user `6a3c9d9bf3c8455aa93ad43b10bfd956`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:25.133258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:25.254253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1157/2290] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:25.266549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:25.268001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:25.490370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a75b0-3620-4ca5-a114-1d6d34833286 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:25.503086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:25.506044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:25.508337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:25.515711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:25.516309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:25.549330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:25.551243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa08c0-5ab8-444f-8e91-ef87735035b3 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:25.582596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1134/2291] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:25.658620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb26c50-f8bf-4477-baaa-9eb98b8bafa1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:25.664023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:25.664832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:25.665225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:25.665759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] PATH_INFO: `/v3/users/c1d97e00acbf45e9b75c13422fb7ec1b` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:25.667784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d97e00acbf45e9b75c13422fb7ec1b)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:25.670378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:25.798819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d97e00acbf45e9b75c13422fb7ec1b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:25.814401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d97e00acbf45e9b75c13422fb7ec1b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:25.815146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d97e00acbf45e9b75c13422fb7ec1b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:25.815778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d97e00acbf45e9b75c13422fb7ec1b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:25.827831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d97e00acbf45e9b75c13422fb7ec1b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:25.856274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1158/2292] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:25 2022] DELETE /identity/v3/users/c1d97e00acbf45e9b75c13422fb7ec1b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:00:25.860594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:25.860727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:25.861004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:25.869770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:25.891406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:25.893212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:26.017545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1135/2293] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:26.055941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:26.059366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:26.066479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:26.150704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0920e3e-c1ae-417f-ae65-733a5a88bbbf admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:26.156124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:26.157022 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:26.157580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:26.158113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:26.170897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:26.172662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:26.203210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1159/2294] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:27.204382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:27.204382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:27.216507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc951e-2c05-4025-83bf-b59d735b29c2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:27.221847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:27.223331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:27.223846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:27.225634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] PATH_INFO: `/v3/projects/acd20feb8ac346f78547e56f06c558e1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:27.228803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd20feb8ac346f78547e56f06c558e1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:27.231319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:27.312014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd20feb8ac346f78547e56f06c558e1', 'request_id': 'req-80801eca-6573-4325-bad4-0baa6d106eb9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.326422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd20feb8ac346f78547e56f06c558e1', 'request_id': 'req-80801eca-6573-4325-bad4-0baa6d106eb9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.346114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd20feb8ac346f78547e56f06c558e1', 'request_id': 'req-80801eca-6573-4325-bad4-0baa6d106eb9'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.372542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] The token cache is being invalidate because project acd20feb8ac346f78547e56f06c558e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:27.374624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.375394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-80801eca-6573-4325-bad4-0baa6d106eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.382324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:27.382996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1136/2295] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:27 2022] DELETE /identity/v3/projects/acd20feb8ac346f78547e56f06c558e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:27.392574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa36cf-651e-41fb-a8dd-05035da4c6a0 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:27.599758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:27.600656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:27.601177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:27.601667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] PATH_INFO: `/v3/users/d53abba75eef430795e046bd4fe07c65` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:27.604885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53abba75eef430795e046bd4fe07c65)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:27.611617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:27.788318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53abba75eef430795e046bd4fe07c65'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.810625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53abba75eef430795e046bd4fe07c65'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.812659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53abba75eef430795e046bd4fe07c65'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.815698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53abba75eef430795e046bd4fe07c65'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.833138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53abba75eef430795e046bd4fe07c65'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:27.866126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1160/2296] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:27 2022] DELETE /identity/v3/users/d53abba75eef430795e046bd4fe07c65 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:29.104831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897c34-c320-490f-86e8-74ef42c18b32 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.105707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897c34-c320-490f-86e8-74ef42c18b32 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.106171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897c34-c320-490f-86e8-74ef42c18b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.130197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:29.131778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:29.147053 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a7897c34-c320-490f-86e8-74ef42c18b32 None None] MFA Rules not processed for user `6a3c9d9bf3c8455aa93ad43b10bfd956`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:29.163775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a7897c34-c320-490f-86e8-74ef42c18b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:29.211388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:29.293627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1eb1-1b3d-4ec4-91a3-4717c75a3495 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:29.302557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1137/2297] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:29.311432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:29.312304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.312744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.313190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] PATH_INFO: `/v3/projects/4bf66e449a2a47bdbc6369d154149442` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.315046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf66e449a2a47bdbc6369d154149442)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:29.317318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:29.388919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf66e449a2a47bdbc6369d154149442', 'request_id': 'req-3047f813-d009-420c-8ccb-a3cef916522c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:29.407882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf66e449a2a47bdbc6369d154149442', 'request_id': 'req-3047f813-d009-420c-8ccb-a3cef916522c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:29.419043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf66e449a2a47bdbc6369d154149442', 'request_id': 'req-3047f813-d009-420c-8ccb-a3cef916522c'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:29.441100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] The token cache is being invalidate because project 4bf66e449a2a47bdbc6369d154149442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:29.441646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:29.442784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3047f813-d009-420c-8ccb-a3cef916522c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:29.445949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1161/2298] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:29 2022] DELETE /identity/v3/projects/4bf66e449a2a47bdbc6369d154149442 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:29.451233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.452016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.452492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.485068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:29.486915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:29.548503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:29.624600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1138/2299] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:29.633579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.633813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.634099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.669759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:29.671534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:29.700480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1162/2300] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:29.704450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:29.717694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c325e5a-52e6-4d33-8b75-e8f23a33adf2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:29.724612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:29.725457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.726027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.726500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.727407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:29.727952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:29.731629 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:29.752381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1139/2301] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:29.756452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:29.766083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e7789-599c-45fe-8313-7a7d79ba0b69 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:29.771830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:29.772838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.773841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.773841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.774483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:29.776969 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:29.814244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1163/2302] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:29 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:29.815634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:29.829251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a8eba-9c81-4471-b547-0e569d658a5d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:29.916019 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:29.918085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.918494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.918881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.919877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:29.922291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:29.958470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:29.968727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1140/2303] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:29 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:29.972366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556a0d-a0a3-49f8-b436-2982606b9370 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:29.979868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:29.980661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:29.981696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:29.981696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:29.982376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:29.984399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:30.001640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1164/2304] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:30.006338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.022684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8a243-ad2e-4c27-b990-f3bc52e25454 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.029394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:30.030500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.031042 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.032039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.032998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:30.036129 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93391683-0b5d-4b3a-b9a5-6701c2cbc4d2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:30.062368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1141/2305] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:30.063109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.088902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443805d1-555c-436d-a736-562d8f66b746 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.099501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:30.099720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.100296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.101121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] PATH_INFO: `/v3/projects/e593a21e5c5b4ca89cf902803b945d80/users/f3d1afd8a25d4cf797d0702d32b4387a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.155030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e593a21e5c5b4ca89cf902803b945d80, user_id=f3d1afd8a25d4cf797d0702d32b4387a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:30.159381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:30.201646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1165/2306] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:30 2022] PUT /identity/v3/projects/e593a21e5c5b4ca89cf902803b945d80/users/f3d1afd8a25d4cf797d0702d32b4387a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:30.212126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4302-5af7-4577-b9ed-ba77d3325add None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.212618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4302-5af7-4577-b9ed-ba77d3325add None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.212699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4302-5af7-4577-b9ed-ba77d3325add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.250645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2afd4302-5af7-4577-b9ed-ba77d3325add None None] MFA Rules not processed for user `f3d1afd8a25d4cf797d0702d32b4387a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:30.252166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2afd4302-5af7-4577-b9ed-ba77d3325add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:30.385771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1142/2307] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:30.390197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.478252 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:30.497339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d177baf-5013-408b-ab18-375da48c24e2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.519520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.523507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.531774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.538297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:30.541609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.542763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.543062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.547416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:30.552733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:30.630401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] MFA Rules not processed for user `33485026d0ed4151abe151d0cdbcd177`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:30.636222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:30.654367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1166/2308] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:30 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:30.655176 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.685726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:30.806474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1143/2309] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:30.830994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.834015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:30.844309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4d73f-ead7-48f6-911f-1cb0ddcb4939 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.858284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:30.858914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.859194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.859466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.873043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:30.883227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:30.939595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1167/2310] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:30 2022] POST /identity/v3/users => generated 298 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:30.944226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:30.964399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df620dfd-94e5-45d5-aa55-260dbf318e0c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.965766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c14b8-e3fa-4f06-a295-b1675375e2ce None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:30.976166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:30.976782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:30.977062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:30.977336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:30.980196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:30.982645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.009334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1168/2311] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:31.011038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:31.028637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a03a-8f7f-48b7-8b40-955aaea0d173 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:31.035473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:31.036567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.036848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.037121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] PATH_INFO: `/v3/projects/b318159da2304a2cbf6dab8f62b2f644/users/60694b6eb6fc4e1baddb0a0445232aed/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.055236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:31.055939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.056304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.057231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.059297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:31.059978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b318159da2304a2cbf6dab8f62b2f644, user_id=60694b6eb6fc4e1baddb0a0445232aed, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:31.063842 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.081425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1169/2312] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:31 2022] PUT /identity/v3/projects/b318159da2304a2cbf6dab8f62b2f644/users/60694b6eb6fc4e1baddb0a0445232aed/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:31.085954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:31.192963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:31.195292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.217328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9f625-0d3a-4f1a-b94a-3487f88a9d04 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:31.220739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1144/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:31.239882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:31.240774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.241283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.241839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.242594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:31.244641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b46ff9-733c-4f85-b2df-3ab821ed4cb9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.265885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1170/2314] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:31.270380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:31.304396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca71dd-9380-494e-a622-4a3f99f29234 service glance] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:31.324797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:31.326130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.326776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.327083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] PATH_INFO: `/v3/projects/b318159da2304a2cbf6dab8f62b2f644/users/60694b6eb6fc4e1baddb0a0445232aed/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.333259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b318159da2304a2cbf6dab8f62b2f644, user_id=60694b6eb6fc4e1baddb0a0445232aed, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:31.339354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.371750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1145/2315] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:31 2022] PUT /identity/v3/projects/b318159da2304a2cbf6dab8f62b2f644/users/60694b6eb6fc4e1baddb0a0445232aed/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:31.387199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e38d5c-1b71-4d88-ae93-d9dc6cb27488 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.387545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e38d5c-1b71-4d88-ae93-d9dc6cb27488 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.387827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e38d5c-1b71-4d88-ae93-d9dc6cb27488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.434298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-52e38d5c-1b71-4d88-ae93-d9dc6cb27488 None None] MFA Rules not processed for user `60694b6eb6fc4e1baddb0a0445232aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:31.435611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-52e38d5c-1b71-4d88-ae93-d9dc6cb27488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:31.476357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:31.615430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1171/2316] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:31.630255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:31.631905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:31.670507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:31.703506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ecc0-6829-418c-acaa-d31651bbd61a None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.703952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ecc0-6829-418c-acaa-d31651bbd61a None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.704278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ecc0-6829-418c-acaa-d31651bbd61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.757424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5b61ecc0-6829-418c-acaa-d31651bbd61a None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:31.759414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5b61ecc0-6829-418c-acaa-d31651bbd61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:31.781110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d269-ea89-4b90-aa12-1fc320185894 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:31.802202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:31.803184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:31.803290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:31.803534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:31.842243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:31.845265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:31.876751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1146/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:31.904718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1172/2318] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:31.993679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:31.996663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:32.056600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:32.062469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:32.062753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:32.099080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] MFA Rules not processed for user `eb1aff5c8217437ea3436ccf9a892054`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:32.100338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:32.154220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73555e3-f09d-4ed4-bd37-31f8bdcabce8 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:32.162173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:32.163154 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:32.163598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:32.164027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:32.176490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:32.178821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:32.206230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1147/2319] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:00:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:00:32.207349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:32.224014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46451b-b784-48da-aff9-20f63cb5d209 service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:32.233155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:32.237925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:32.238601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:32.239111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:32.240440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1173/2320] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:32.361036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:32.362760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071cf97-b5f0-4f3d-a5dd-cc20c7d591b4 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:32.392250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1148/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:35.982916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:36.191210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a6906-61b2-4794-898a-e9bb8d18a6a4 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:36.221651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:36.222371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:36.222639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:36.222913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] PATH_INFO: `/v3/users/f3d1afd8a25d4cf797d0702d32b4387a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:36.247010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d1afd8a25d4cf797d0702d32b4387a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:36.248988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:36.270017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:36.424726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d1afd8a25d4cf797d0702d32b4387a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:36.449904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d1afd8a25d4cf797d0702d32b4387a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:36.453506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d1afd8a25d4cf797d0702d32b4387a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:36.455602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d1afd8a25d4cf797d0702d32b4387a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:36.475919 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d1afd8a25d4cf797d0702d32b4387a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:36.527515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1174/2322] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:35 2022] DELETE /identity/v3/users/f3d1afd8a25d4cf797d0702d32b4387a => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:36.532730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:36.533258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:36.533258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:36.573884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:36.575142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:36.674628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:36.727043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1149/2323] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:36.791837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:36.793297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:36.966240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7062-2ffe-42c5-9783-7e6ba1b4f637 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:36.972816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:36.973604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:36.974000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:36.979602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:37.000580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:37.002339 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:37.030659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1175/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:37.964667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:37.968714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:37.985726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33727e-dc4b-4924-907f-140e549a81f9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:37.992642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:37.994034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:37.994641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:37.995331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] PATH_INFO: `/v3/projects/e593a21e5c5b4ca89cf902803b945d80` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:37.997962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e593a21e5c5b4ca89cf902803b945d80)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:38.000120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:38.087798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e593a21e5c5b4ca89cf902803b945d80', 'request_id': 'req-642097e8-2026-4384-8afa-d2d511c9bd6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.100431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e593a21e5c5b4ca89cf902803b945d80', 'request_id': 'req-642097e8-2026-4384-8afa-d2d511c9bd6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.113509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e593a21e5c5b4ca89cf902803b945d80', 'request_id': 'req-642097e8-2026-4384-8afa-d2d511c9bd6e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.140470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] The token cache is being invalidate because project e593a21e5c5b4ca89cf902803b945d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:38.140915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.141590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.143501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1150/2325] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:37 2022] DELETE /identity/v3/projects/e593a21e5c5b4ca89cf902803b945d80 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:38.147428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:38.160470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7b615-2599-443e-a690-81e06db9315d service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:38.224531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:38.225458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:38.225927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:38.226354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] PATH_INFO: `/v3/users/60694b6eb6fc4e1baddb0a0445232aed` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:38.227641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60694b6eb6fc4e1baddb0a0445232aed)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:38.229772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:38.335793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:38.408850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60694b6eb6fc4e1baddb0a0445232aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.425037 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60694b6eb6fc4e1baddb0a0445232aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.425998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60694b6eb6fc4e1baddb0a0445232aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.426797 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60694b6eb6fc4e1baddb0a0445232aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.463522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-88eab47c-f016-43dd-b73b-ab2813ce32e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60694b6eb6fc4e1baddb0a0445232aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:38.537888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1176/2326] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:38 2022] DELETE /identity/v3/users/60694b6eb6fc4e1baddb0a0445232aed => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:39.123244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.126660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:39.151581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642097e8-2026-4384-8afa-d2d511c9bd6e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.235283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.235891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.236139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.236396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] PATH_INFO: `/v3/projects/b318159da2304a2cbf6dab8f62b2f644` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.237536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b318159da2304a2cbf6dab8f62b2f644)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.239520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.306798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b318159da2304a2cbf6dab8f62b2f644', 'request_id': 'req-27dc6a01-fccb-4e0d-a285-f96913537b18'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:39.317060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b318159da2304a2cbf6dab8f62b2f644', 'request_id': 'req-27dc6a01-fccb-4e0d-a285-f96913537b18'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:39.329076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b318159da2304a2cbf6dab8f62b2f644', 'request_id': 'req-27dc6a01-fccb-4e0d-a285-f96913537b18'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:39.335164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:39.345918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] The token cache is being invalidate because project b318159da2304a2cbf6dab8f62b2f644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:39.346556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:39.347595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-27dc6a01-fccb-4e0d-a285-f96913537b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:39.349420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1151/2327] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:39 2022] DELETE /identity/v3/projects/b318159da2304a2cbf6dab8f62b2f644 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:39.354887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.355423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.355949 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.378044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:39.379641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:39.465944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1177/2328] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:39.475272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.475666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.476044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.499185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:39.500365 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:39.511350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1152/2329] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:39.515821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.530395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db7a12b-e5cd-4d22-a402-a5fe0750dab0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.535572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.536194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.536468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.537055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.537896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:39.538345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.540685 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.573790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1178/2330] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 08:00:39.575030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.590070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213db5e-8168-4fee-a6c4-5eb68d6c26fb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.604929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.605656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.607190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.607190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.608044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.613148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.662331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.664047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1153/2331] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:39 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:39.678086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea14e-fd36-4275-8018-488d33c65241 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.767448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.768361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.768858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.769296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.770244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.772337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.810184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1179/2332] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:39 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:39.814100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.831170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5199f08-5a59-43d3-ba04-2e711581ef44 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.840627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.842130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.842693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.843208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.844223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.846169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.860004 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1154/2333] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:39.863524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.879385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b23d4-5cf6-4c73-a0c8-f4db37e0194c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.889977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.890616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.890908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.891182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.891876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.893453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dbb3c-f7b2-492d-9077-f606505d1cf2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.930545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1180/2334] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:39.935148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:39.948204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed28d6b-d026-419a-890b-d501d99589c8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:39.955681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:39.957164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:39.957667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:39.958183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] PATH_INFO: `/v3/projects/74d1aed5829d4ef7a607bf953a0014a1/users/d1994a8fb5c34844aec31bca40a86e2d/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:39.976359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d1aed5829d4ef7a607bf953a0014a1, user_id=d1994a8fb5c34844aec31bca40a86e2d, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:39.979466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:39.999058 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1155/2335] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:39 2022] PUT /identity/v3/projects/74d1aed5829d4ef7a607bf953a0014a1/users/d1994a8fb5c34844aec31bca40a86e2d/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:40.012048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.012326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.012593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.035353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:40.068074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] MFA Rules not processed for user `d1994a8fb5c34844aec31bca40a86e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:40.069880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:40.195764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.197794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1181/2336] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:40.294670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293dbb-efc1-42ea-b024-9acd3bb13f3e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.300807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.301442 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.301721 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.302169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.303147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:40.305803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:40.389765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1156/2337] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:40 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:40.391735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.412547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2627e-263c-4afb-9754-cadab080c391 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.559393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.559945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.560850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.561140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.561835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:40.567184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:40.602997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1182/2338] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:40 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:40.604188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.619607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4aba0-ba75-47d4-bfc1-a5b1ecc30a75 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.627794 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.628846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.629249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.629661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.630714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:40.635778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:40.670353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1157/2339] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:40.674522 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.686268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23521a8c-16ef-4d25-8bc9-68f3526bb932 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.691537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.692224 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.694752 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.695071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] PATH_INFO: `/v3/projects/0c42c529aa4a4106b0777017a388d6cf/users/58f634ccb06c41ccb4b85c004599a868/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.712083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c42c529aa4a4106b0777017a388d6cf, user_id=58f634ccb06c41ccb4b85c004599a868, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:40.715950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:40.748288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1183/2340] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:40 2022] PUT /identity/v3/projects/0c42c529aa4a4106b0777017a388d6cf/users/58f634ccb06c41ccb4b85c004599a868/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:40.752324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.772996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfad3f-bb65-46c3-bc9a-c922bb6b512d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.926255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.927027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.929393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.929789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:40.930563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:40.932440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6a04-2052-4984-bdf5-e357e03d53e9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:40.960413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1158/2341] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:40.964761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:40.979970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f260d-bb62-41b4-ba11-341ce9a67330 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:40.994722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:40.995432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:40.995682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:40.995954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] PATH_INFO: `/v3/projects/0c42c529aa4a4106b0777017a388d6cf/users/58f634ccb06c41ccb4b85c004599a868/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:41.004257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c42c529aa4a4106b0777017a388d6cf, user_id=58f634ccb06c41ccb4b85c004599a868, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:41.007962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:41.061094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:41.061390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:41.061649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:41.066182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1184/2342] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:40 2022] PUT /identity/v3/projects/0c42c529aa4a4106b0777017a388d6cf/users/58f634ccb06c41ccb4b85c004599a868/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:41.070475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:41.100206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] MFA Rules not processed for user `58f634ccb06c41ccb4b85c004599a868`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:41.101421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:41.269000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1159/2343] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:41.281532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:41.283309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:41.451085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84960-2d5d-44fa-b5e4-2d405d7dbf3b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:41.460593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:41.461991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:41.463357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:41.463968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:41.478124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:41.490860 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:41.493754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27508865-aed5-47da-a4ee-2a7c6e82d823 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:41.524013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1185/2344] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:42.439067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:42.531966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b800d46-ce91-468d-ae1b-2bafb5d8df58 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:42.539876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:42.541100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:42.541391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:42.541666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] PATH_INFO: `/v3/users/d1994a8fb5c34844aec31bca40a86e2d` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:42.544303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1994a8fb5c34844aec31bca40a86e2d)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:42.546234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:42.640078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1994a8fb5c34844aec31bca40a86e2d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:42.656566 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1994a8fb5c34844aec31bca40a86e2d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:42.657844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1994a8fb5c34844aec31bca40a86e2d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:42.658805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1994a8fb5c34844aec31bca40a86e2d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:42.668474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1994a8fb5c34844aec31bca40a86e2d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:42.719016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1160/2345] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:42 2022] DELETE /identity/v3/users/d1994a8fb5c34844aec31bca40a86e2d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:42.720639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:42.721114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:42.721386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:42.750496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:42.753378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:42.906849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1186/2346] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:42.927275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:42.928862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:42.960796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:43.120527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29091b-2c25-422c-9db8-d510579bdf44 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:43.126748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:43.127379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:43.127666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:43.127942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:43.143745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:43.145234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:43.157604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1161/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:43.644945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:43.646412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:43.678499 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c61362-d74a-4d47-844c-8bc0f05c1c74 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:43.685381 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:43.687374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:43.687972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:43.688510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] PATH_INFO: `/v3/projects/74d1aed5829d4ef7a607bf953a0014a1` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:43.689966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d1aed5829d4ef7a607bf953a0014a1)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:43.692180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:43.864262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d1aed5829d4ef7a607bf953a0014a1', 'request_id': 'req-67f89cfd-6843-436e-aa53-03e7d99cda91'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:43.903195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d1aed5829d4ef7a607bf953a0014a1', 'request_id': 'req-67f89cfd-6843-436e-aa53-03e7d99cda91'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:43.923179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d1aed5829d4ef7a607bf953a0014a1', 'request_id': 'req-67f89cfd-6843-436e-aa53-03e7d99cda91'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:43.956956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] The token cache is being invalidate because project 74d1aed5829d4ef7a607bf953a0014a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:43.957255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:43.957951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:43.960962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1187/2348] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:43 2022] DELETE /identity/v3/projects/74d1aed5829d4ef7a607bf953a0014a1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:43.964910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:43.976807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055323-257f-48f1-b78d-3f8ba138cf21 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:44.070316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:44.070966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:44.071247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:44.071524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] PATH_INFO: `/v3/users/58f634ccb06c41ccb4b85c004599a868` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:44.073628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f634ccb06c41ccb4b85c004599a868)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:44.075723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:44.152354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f634ccb06c41ccb4b85c004599a868'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.160418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f634ccb06c41ccb4b85c004599a868'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.161161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f634ccb06c41ccb4b85c004599a868'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.161876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f634ccb06c41ccb4b85c004599a868'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.172363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-269e0dcf-0043-4392-95c8-330adc40df3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f634ccb06c41ccb4b85c004599a868'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.204139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1162/2349] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:43 2022] DELETE /identity/v3/users/58f634ccb06c41ccb4b85c004599a868 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:44.739721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:44.745948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:44.776973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f89cfd-6843-436e-aa53-03e7d99cda91 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:44.863802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:44.864479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:44.865119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:44.865119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] PATH_INFO: `/v3/projects/0c42c529aa4a4106b0777017a388d6cf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:44.866710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c42c529aa4a4106b0777017a388d6cf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:44.869884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:44.915802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:44.957365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c42c529aa4a4106b0777017a388d6cf', 'request_id': 'req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.964844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c42c529aa4a4106b0777017a388d6cf', 'request_id': 'req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.974047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c42c529aa4a4106b0777017a388d6cf', 'request_id': 'req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.987715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] The token cache is being invalidate because project 0c42c529aa4a4106b0777017a388d6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:44.987992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.988865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-dcb69d6d-95db-4f5e-afd5-5d8c8d6b2ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:44.991310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1188/2350] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:44 2022] DELETE /identity/v3/projects/0c42c529aa4a4106b0777017a388d6cf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:44.996209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:44.997047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:44.997484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.031029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:45.046245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:45.112489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:45.194553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1163/2351] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:45.204407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.205058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.205495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.253432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:45.255508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:45.267437 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1189/2352] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:45.271078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.288565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e569e-931c-477f-9f84-c58b1eae1ef7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.295413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.296103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.296268 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.296545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.297283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:45.297690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.299493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.313437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1164/2353] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:45.317835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.331055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0371212-8633-4bf2-8f23-72347e25993b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.335827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.336630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.336920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.338455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.340664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.342828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.383750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1190/2354] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:45 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:45.388702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.400630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364e4d0-3800-4bfb-a003-82ff9019e757 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.470711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.471471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.471849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.472630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.473119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.475205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.507404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1165/2355] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:45 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:45.508429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.523195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e5b29-4cb3-4f0c-9633-4bba67c99fd9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.531184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.531184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.531184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.531570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.532090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.533779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.546848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1191/2356] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:45.551568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.580013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757d39-d70c-4709-b42d-f4d9fa12e9e8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.586906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.587835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.588357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.589459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.590726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.595297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba79be-9888-4d6e-98e2-5defd63837d6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.618686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1166/2357] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:45.621380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.636468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe81f-27fe-4492-9757-c75fcb90bd9a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:45.648621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:45.649298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.649576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.649873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] PATH_INFO: `/v3/projects/d0bc2edb1a074d46ab32639a6eb59562/users/a088f29aaebf4138a9b0a2ce871bc82b/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.701229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bc2edb1a074d46ab32639a6eb59562, user_id=a088f29aaebf4138a9b0a2ce871bc82b, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:45.704281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:45.725165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1192/2358] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:45 2022] PUT /identity/v3/projects/d0bc2edb1a074d46ab32639a6eb59562/users/a088f29aaebf4138a9b0a2ce871bc82b/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:45.739063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:45.740082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:45.740607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:45.808736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] MFA Rules not processed for user `a088f29aaebf4138a9b0a2ce871bc82b`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:45.810404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:45.969581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1167/2359] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:45.982800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:45.986015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:46.123211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbca3b-9440-4e59-aaa8-f8d58776f3d1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:46.136760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:46.137836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:46.138347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:46.138912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:46.153306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:46.155623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003d970-0295-448c-bc7d-6644ad37e9b8 service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:46.171663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1193/2360] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:46.255808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:46.257983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:46.307657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:46.413003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add1cc2-aeba-465a-a858-01cf50574be9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:46.417853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:46.418504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:46.418784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:46.419056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] PATH_INFO: `/v3/users/5b1aaa91cad34cdc9fe908f07d40b123` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:46.420929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1aaa91cad34cdc9fe908f07d40b123)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:46.422950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:46.517520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1aaa91cad34cdc9fe908f07d40b123'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:46.531959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1aaa91cad34cdc9fe908f07d40b123'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:46.532940 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1aaa91cad34cdc9fe908f07d40b123'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:46.533757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1aaa91cad34cdc9fe908f07d40b123'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:46.555997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1aaa91cad34cdc9fe908f07d40b123'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:46.597903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1168/2361] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:00:46 2022] DELETE /identity/v3/users/5b1aaa91cad34cdc9fe908f07d40b123 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:46.602390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:46.602675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:46.602951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:46.626551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:46.631445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:46.702269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:46.745317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1194/2362] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:46.756023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:46.759386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:46.881620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec4bd-80c0-481b-a411-db9e8b5775ed admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:46.886553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:46.887462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:46.887924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:46.888386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:46.908766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:46.917825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc34be5-4183-48ad-8bf9-d3486ce98462 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:46.952168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1169/2363] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:47.632798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:47.637096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:47.653405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22e7b7-6cc0-4166-886e-1480f8f6ae50 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:47.671453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:47.673238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:47.673858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:47.674728 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] PATH_INFO: `/v3/projects/d5e85beab3174ee9bfa72c4f6dcebccc` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:47.678396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e85beab3174ee9bfa72c4f6dcebccc)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:47.690372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:47.776838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e85beab3174ee9bfa72c4f6dcebccc', 'request_id': 'req-a7a99c2d-469c-4662-8a2f-c6e25b07407b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:47.787354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e85beab3174ee9bfa72c4f6dcebccc', 'request_id': 'req-a7a99c2d-469c-4662-8a2f-c6e25b07407b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:47.807287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e85beab3174ee9bfa72c4f6dcebccc', 'request_id': 'req-a7a99c2d-469c-4662-8a2f-c6e25b07407b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:47.829574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] The token cache is being invalidate because project d5e85beab3174ee9bfa72c4f6dcebccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:00:47.829907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:47.841756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a7a99c2d-469c-4662-8a2f-c6e25b07407b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:00:47.841756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1195/2364] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:00:47 2022] DELETE /identity/v3/projects/d5e85beab3174ee9bfa72c4f6dcebccc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:47.842126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:47.842126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:47.842126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:47.862085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:47.863460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:47.935569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:48.011184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1170/2365] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:48.012457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.012795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.013458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.041622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:48.044095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:48.059076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1196/2366] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:48.063801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.081225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c6e6a-512d-4464-a894-ec1aed054fa4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.086309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.086922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.087201 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.088017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.088681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:00:48.088982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.090620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.115116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.118813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1171/2367] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:00:48.126006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e95601-da54-4dcb-b336-313dc71d340f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.131132 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.131748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.132026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.132301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.132947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.135150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.180673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1197/2368] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:00:48 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:00:48.183429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.204494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f49b-3c55-466e-84e7-18da698eb0e4 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.290007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.290708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.290989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.291262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.291911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.294263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.330120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1172/2369] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:00:48 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:00:48.334406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.349996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df309b3-a5ba-4df1-bdbb-8802336e5e81 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.359071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.360361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.361028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.361929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.364283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.367134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.379636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1198/2370] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:00:48.383829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.401121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b14c5-dd14-4a1e-81a9-f5f35463f2dd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.424483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.425684 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.426223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.426760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.427671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.429579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882ecc9-cfd5-4b0c-944e-44206e4f557d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.449527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1173/2371] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:00:48.454730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.466886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6b7c-cd79-4919-90a4-4d5e6a999f15 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.480060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.480787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.481067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.481338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] PATH_INFO: `/v3/projects/6e18d3f916e34af2adc82dad26e17987/users/01e09ce81d22442a9c672c6e634c5013/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.501165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e18d3f916e34af2adc82dad26e17987, user_id=01e09ce81d22442a9c672c6e634c5013, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.505490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.531168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1199/2372] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:00:48 2022] PUT /identity/v3/projects/6e18d3f916e34af2adc82dad26e17987/users/01e09ce81d22442a9c672c6e634c5013/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:00:48.542938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cca92-26c3-4e59-93d8-94ad4a09855b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.543709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cca92-26c3-4e59-93d8-94ad4a09855b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.543709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cca92-26c3-4e59-93d8-94ad4a09855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.581134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-394cca92-26c3-4e59-93d8-94ad4a09855b None None] MFA Rules not processed for user `01e09ce81d22442a9c672c6e634c5013`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:48.582409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-394cca92-26c3-4e59-93d8-94ad4a09855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:48.690896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1174/2373] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:48.702805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:48.704873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:48.776981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.777264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.778055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.789791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d8a8a-af1a-4314-9a79-5c78f9e0fbb6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:48.789791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:48.789791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:48.789791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:48.790269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:48.800043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] MFA Rules not processed for user `67ec6b7d73494d9bb0294c21277407ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:48.800647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:48.801248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:48.802265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a9075-4749-4c12-aa3e-9614592d19bc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:48.812592 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1200/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:00:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:00:48.845405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:48.874844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1175/2375] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:00:52.893662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e58d7a-c1e3-43af-91cf-9b8557742c27 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:52.894241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e58d7a-c1e3-43af-91cf-9b8557742c27 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:52.894241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e58d7a-c1e3-43af-91cf-9b8557742c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:52.918475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-27e58d7a-c1e3-43af-91cf-9b8557742c27 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:52.920067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-27e58d7a-c1e3-43af-91cf-9b8557742c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:52.925415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:00:53.030557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1201/2376] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:00:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:00:53.037461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:00:53.039596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:53.210006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de29d8f-1424-4a39-8fc1-6f70133951ae None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:00:53.232840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:00:53.233487 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:53.233814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:53.234124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:53.253315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:00:53.255320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:00:53.276926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1176/2377] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:00:53.886256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713823c-7ef1-43ef-938c-9a016d78cb37 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:00:53.886831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713823c-7ef1-43ef-938c-9a016d78cb37 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:00:53.887268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713823c-7ef1-43ef-938c-9a016d78cb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:00:53.921223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e713823c-7ef1-43ef-938c-9a016d78cb37 None None] MFA Rules not processed for user `272f5244f9724e00bd35dd09fedc6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:00:53.922489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e713823c-7ef1-43ef-938c-9a016d78cb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:00:53.990869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1202/2378] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:00.124581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:00.193043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ea875-fc96-4b25-832f-3bb3c0e8a1e2 service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:00.197875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:00.198512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:00.198855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:00.199570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] PATH_INFO: `/v3/users/01e09ce81d22442a9c672c6e634c5013` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:00.200472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e09ce81d22442a9c672c6e634c5013)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:00.202607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:00.268641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e09ce81d22442a9c672c6e634c5013'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:00.280361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e09ce81d22442a9c672c6e634c5013'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:00.281171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e09ce81d22442a9c672c6e634c5013'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:00.281890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e09ce81d22442a9c672c6e634c5013'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:00.298732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e09ce81d22442a9c672c6e634c5013'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:00.318275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:00.335038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1177/2379] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:01:00 2022] DELETE /identity/v3/users/01e09ce81d22442a9c672c6e634c5013 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:00.336831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:00.337948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:00.338052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:00.363851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:00.365208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:00.464392 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1203/2380] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:01:00.474085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:00.475505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:00.539029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0949d6-3375-40fa-83cf-71876d6bcd10 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:00.544553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:00.545174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:00.545554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:00.545863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:00.563670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:00.565282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723729-f9e4-43c2-823a-420c6618c458 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:00.577653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1178/2381] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:01:00.889937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:00.891573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:00.907982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b7fad-1902-4215-9715-68d2be1fb5ff None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:00.913639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:00.914546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:00.915622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:00.916068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] PATH_INFO: `/v3/projects/6e18d3f916e34af2adc82dad26e17987` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:00.917507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e18d3f916e34af2adc82dad26e17987)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:00.927341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:01.030453 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e18d3f916e34af2adc82dad26e17987', 'request_id': 'req-9053cad0-c4fe-41be-b720-992ae17271f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:01.043150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e18d3f916e34af2adc82dad26e17987', 'request_id': 'req-9053cad0-c4fe-41be-b720-992ae17271f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:01.045247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:01.057502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e18d3f916e34af2adc82dad26e17987', 'request_id': 'req-9053cad0-c4fe-41be-b720-992ae17271f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:01.078580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] The token cache is being invalidate because project 6e18d3f916e34af2adc82dad26e17987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:01:01.079119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:01.080043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:01.082137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1204/2382] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:01:00 2022] DELETE /identity/v3/projects/6e18d3f916e34af2adc82dad26e17987 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:10.573342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=28 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:11.073634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:15.877262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:15.877720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:15.877976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:15.898222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:15.899956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:15.962613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1179/2383] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:01:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:15.975976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:15.978611 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:16.065687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cad0-c4fe-41be-b720-992ae17271f2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:16.068316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:16.068932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:16.069217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:16.069497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:16.078709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:16.080288 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642a374-9ede-40aa-a210-c90bf3c8c349 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:16.089232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1205/2384] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:19.449559 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:19.451180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:19.536648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030d440-88e2-4d2f-9550-8ea6b5ca9b62 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:19.542224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:19.542893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:19.543194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:19.543470 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] PATH_INFO: `/v3/users/a088f29aaebf4138a9b0a2ce871bc82b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:19.544863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a088f29aaebf4138a9b0a2ce871bc82b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:19.547648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:19.613463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a088f29aaebf4138a9b0a2ce871bc82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:19.623732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a088f29aaebf4138a9b0a2ce871bc82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:19.625367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a088f29aaebf4138a9b0a2ce871bc82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:19.626508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a088f29aaebf4138a9b0a2ce871bc82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:19.638616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a088f29aaebf4138a9b0a2ce871bc82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:19.662302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1180/2385] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:01:19 2022] DELETE /identity/v3/users/a088f29aaebf4138a9b0a2ce871bc82b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:01:19.666864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:19.667393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:19.667393 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:19.695794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:19.698488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:19.808278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1206/2386] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:19.815547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:19.818091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:19.851628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:19.891743 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9ea27-75db-4422-867b-f37580d686e5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:19.895982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:19.896885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:19.897319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:19.897786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:19.914926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:19.916450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb3a74-e2e3-4753-971a-3228420d7d69 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:19.937814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1181/2387] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:01:20.535409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:20.538092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:20.575471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1d4e6-ea2c-4a6a-b6ec-2bf2c33a0593 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:20.591372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:20.591831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:20.592115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:20.592973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] PATH_INFO: `/v3/projects/d0bc2edb1a074d46ab32639a6eb59562` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:20.594862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bc2edb1a074d46ab32639a6eb59562)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:20.598798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:20.680682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bc2edb1a074d46ab32639a6eb59562', 'request_id': 'req-206cdb35-d6c8-466e-ae75-773aecca9e42'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:20.693464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bc2edb1a074d46ab32639a6eb59562', 'request_id': 'req-206cdb35-d6c8-466e-ae75-773aecca9e42'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:20.710951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bc2edb1a074d46ab32639a6eb59562', 'request_id': 'req-206cdb35-d6c8-466e-ae75-773aecca9e42'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:20.741247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] The token cache is being invalidate because project d0bc2edb1a074d46ab32639a6eb59562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:01:20.741549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:20.744724 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-206cdb35-d6c8-466e-ae75-773aecca9e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:20.747200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1207/2388] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:01:20 2022] DELETE /identity/v3/projects/d0bc2edb1a074d46ab32639a6eb59562 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:20.752056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b87f10-5901-448c-9a85-019c5261424f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:20.752513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b87f10-5901-448c-9a85-019c5261424f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:20.752599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b87f10-5901-448c-9a85-019c5261424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:20.799275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-10b87f10-5901-448c-9a85-019c5261424f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:20.812746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-10b87f10-5901-448c-9a85-019c5261424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:20.993774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1182/2389] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:21.062515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.062990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.063357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.107317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:21.110197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:21.136428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1208/2390] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:21.140880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b87f10-5901-448c-9a85-019c5261424f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.157407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b87f10-5901-448c-9a85-019c5261424f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.181573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.182771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.183385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.183659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.184446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:01:21.185029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.188997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.227546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1183/2391] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:01:21.233175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.250083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe0b2f-e566-4acc-a80e-c791edc644b5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.260744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.262751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.264595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.265044 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.265931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.268672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.279018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:21.319025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1209/2392] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:01:21 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:01:21.323207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.342942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb5d81-e294-4315-a9a8-2906179b3fb0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.490692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.491349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.491778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.492100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.493548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.506189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.532173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1184/2393] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:01:21 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:01:21.536194 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.547358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1df44-4978-463c-a305-4eebdf442fea admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.552301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.552914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.553200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.553505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.554346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.556042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.572993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.579360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1210/2394] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:01:21.584168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc9845-14a5-40b6-aaed-f0392c263278 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.600410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.601141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.601485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.601798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.603110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.604731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b2a9-d86a-429e-96cb-e2e3490d08bd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.621816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1185/2395] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:01:21.622016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.633020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b25abe-9bcc-4d5e-82c7-c7c13d3c7c1b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.640446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.641326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.641715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.642270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] PATH_INFO: `/v3/projects/801e2d6bb5634e2e98f47fb4f16bf765/users/a80688708cfd4cd99fd9032fa27869b8/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.662918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] RBAC: Authorizing `identity:create_grant(project_id=801e2d6bb5634e2e98f47fb4f16bf765, user_id=a80688708cfd4cd99fd9032fa27869b8, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.667032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.687362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1211/2396] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:01:21 2022] PUT /identity/v3/projects/801e2d6bb5634e2e98f47fb4f16bf765/users/a80688708cfd4cd99fd9032fa27869b8/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:21.698585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171254d-6870-4793-a252-20a0e691eaec None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.698881 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171254d-6870-4793-a252-20a0e691eaec None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.699229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171254d-6870-4793-a252-20a0e691eaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.732067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7171254d-6870-4793-a252-20a0e691eaec None None] MFA Rules not processed for user `a80688708cfd4cd99fd9032fa27869b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:21.733393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7171254d-6870-4793-a252-20a0e691eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:21.755927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:21.829178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1186/2397] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:21.840183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:21.842042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:21.913861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3b0f4-6412-4a64-9643-f51e925c582b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:21.918092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:21.918879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:21.919180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:21.919470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:21.928629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:21.930906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadf176-0a6d-4b7c-8c86-b45986139169 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:21.940094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1212/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:01:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:01:23.404074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:23.404697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:23.404978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:23.428101 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] MFA Rules not processed for user `a80688708cfd4cd99fd9032fa27869b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:23.429351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:23.438996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1187/2399] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:31.936003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:33.421110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=20 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:34.912341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:34.912840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:34.912960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:34.932801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:34.934010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:34.989043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1213/2400] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:01:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:35.002168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:35.003541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:35.071545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15009820-2849-4b44-a8a1-a0dc5830ac79 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:35.076774 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:35.077381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:35.077659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:35.077970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:35.085786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:35.087390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:35.095352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1188/2401] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:37.239559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:37.241002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:37.306938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eef176-cfe7-4953-bdc4-77f74ad6eb8c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:37.311857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:37.312525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:37.312800 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:37.313070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] PATH_INFO: `/v3/users/eb1aff5c8217437ea3436ccf9a892054` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:37.314228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1aff5c8217437ea3436ccf9a892054)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:37.316185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:37.363968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1aff5c8217437ea3436ccf9a892054'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.371349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1aff5c8217437ea3436ccf9a892054'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.371958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1aff5c8217437ea3436ccf9a892054'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.372558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1aff5c8217437ea3436ccf9a892054'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.379565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1aff5c8217437ea3436ccf9a892054'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.396951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1214/2402] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:01:37 2022] DELETE /identity/v3/users/eb1aff5c8217437ea3436ccf9a892054 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:37.575884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:37.591421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cdf24-0216-464f-9b27-3d4b059b386c service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:37.679178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:37.679905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:37.680261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:37.680608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] PATH_INFO: `/v3/projects/44a6ed00e5ad40be9c50d72808559c8e` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:37.682523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a6ed00e5ad40be9c50d72808559c8e)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:37.684630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:37.755688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a6ed00e5ad40be9c50d72808559c8e', 'request_id': 'req-4fb9fa49-aa3f-427a-a5a6-9796b0298575'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.764918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a6ed00e5ad40be9c50d72808559c8e', 'request_id': 'req-4fb9fa49-aa3f-427a-a5a6-9796b0298575'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.776455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a6ed00e5ad40be9c50d72808559c8e', 'request_id': 'req-4fb9fa49-aa3f-427a-a5a6-9796b0298575'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.789869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] The token cache is being invalidate because project 44a6ed00e5ad40be9c50d72808559c8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:01:37.790323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.790984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4fb9fa49-aa3f-427a-a5a6-9796b0298575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:37.804420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1189/2403] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:01:37 2022] DELETE /identity/v3/projects/44a6ed00e5ad40be9c50d72808559c8e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:38.004380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:38.005812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:38.033274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:38.063492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd192cf-4412-472d-a071-7533a42ed67c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:38.111510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:38.112104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:38.112376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:38.112640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:38.114350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:38.176972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:38.178441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:38.187820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1215/2404] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:01:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:01:44.222648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436619c-8369-4dad-8123-9c9b1f668842 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:44.223113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436619c-8369-4dad-8123-9c9b1f668842 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:44.223217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436619c-8369-4dad-8123-9c9b1f668842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:44.242923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-7436619c-8369-4dad-8123-9c9b1f668842 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:44.244131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7436619c-8369-4dad-8123-9c9b1f668842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:44.257334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:44.301352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1190/2405] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:01:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:44.313332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:44.314730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:44.383377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b591e-9ac7-45fb-956b-a6020dd0f2d7 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:44.388467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:44.389114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:44.389415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:44.389690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:44.397805 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:44.399903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:44.407965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1216/2406] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:44.918122 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef331f-b09f-419f-8d69-0cb659e61b77 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:44.918411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef331f-b09f-419f-8d69-0cb659e61b77 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:44.918685 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef331f-b09f-419f-8d69-0cb659e61b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:44.947934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0fef331f-b09f-419f-8d69-0cb659e61b77 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:44.949125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0fef331f-b09f-419f-8d69-0cb659e61b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:44.956699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1191/2407] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:01:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:44.970066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:44.981302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ccd3c-7854-457b-9f31-bf025d257d24 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:44.986330 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:44.986925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:44.987201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:44.987470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:44.996790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:44.998419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:45.008578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1217/2408] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:45.242761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a515-b6f5-492b-bdbf-1d80f69acdfa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:45.243235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a515-b6f5-492b-bdbf-1d80f69acdfa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:45.243360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a515-b6f5-492b-bdbf-1d80f69acdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:45.263109 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-05d8a515-b6f5-492b-bdbf-1d80f69acdfa None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:45.264301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-05d8a515-b6f5-492b-bdbf-1d80f69acdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:45.271515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1192/2409] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:45.283924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:45.295078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5416e-d70c-4572-a4cb-c2b4b50232e8 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:45.300084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:45.300675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:45.300956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:45.301231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:45.307793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:45.312282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:45.314243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:45.325316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1218/2410] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:45.643661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:45.644062 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:45.644560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:45.667902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:45.669106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:45.676093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1193/2411] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:45.688064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:45.699188 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd0cc5-e4ea-4d40-9312-06f50677cef4 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:45.704065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:45.704647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:45.704923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:45.705190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:45.714071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:45.715794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e1493-c3e6-4628-af74-903a8c7acd5e service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:45.723882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1219/2412] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:01:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:01:47.797267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:47.798714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:47.865303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4000-8029-4bfd-b09e-534df52b94b6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:47.870560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:47.871217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:47.871494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:47.871763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] PATH_INFO: `/v3/users/a80688708cfd4cd99fd9032fa27869b8` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:47.872898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80688708cfd4cd99fd9032fa27869b8)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:47.874855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:47.925692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80688708cfd4cd99fd9032fa27869b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:47.934061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80688708cfd4cd99fd9032fa27869b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:47.934681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80688708cfd4cd99fd9032fa27869b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:47.935291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80688708cfd4cd99fd9032fa27869b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:47.942489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80688708cfd4cd99fd9032fa27869b8'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:47.961351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1194/2413] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:01:47 2022] DELETE /identity/v3/users/a80688708cfd4cd99fd9032fa27869b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:47.965967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:47.966443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:47.966540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:47.986767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:47.988472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:48.064231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1220/2414] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:01:48.075625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:48.077069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:48.106073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:48.149949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1e09f-3fb0-44c1-b9fd-cc6603652f18 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:48.153266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:48.154383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:48.155134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:48.155730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:48.168005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:48.169607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb796fb-3a27-44f7-9e3f-c5a7f7c8b753 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:48.180659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1195/2415] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:01:48.358725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:01:48.360355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:48.372756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbd70c-1ee5-4d67-b153-d3a8f867fce9 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:01:48.377335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:01:48.378012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:48.378295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:48.378563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] PATH_INFO: `/v3/projects/801e2d6bb5634e2e98f47fb4f16bf765` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:48.379683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801e2d6bb5634e2e98f47fb4f16bf765)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:01:48.381594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:01:48.428743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801e2d6bb5634e2e98f47fb4f16bf765', 'request_id': 'req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:48.435035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801e2d6bb5634e2e98f47fb4f16bf765', 'request_id': 'req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:48.441873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801e2d6bb5634e2e98f47fb4f16bf765', 'request_id': 'req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:48.453410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] The token cache is being invalidate because project 801e2d6bb5634e2e98f47fb4f16bf765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:01:48.453788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:48.454418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:01:48.456013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1221/2416] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:01:48 2022] DELETE /identity/v3/projects/801e2d6bb5634e2e98f47fb4f16bf765 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:01:58.175687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=15 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:58.450667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:01:58.516787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e45d6-0b8e-4a89-b0b8-414e4f588af2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:01:58.517125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e45d6-0b8e-4a89-b0b8-414e4f588af2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:01:58.517396 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e45d6-0b8e-4a89-b0b8-414e4f588af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:01:58.536385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b77e45d6-0b8e-4a89-b0b8-414e4f588af2 None None] MFA Rules not processed for user `6a3c9d9bf3c8455aa93ad43b10bfd956`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:01:58.537562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b77e45d6-0b8e-4a89-b0b8-414e4f588af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:01:58.591957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1196/2417] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:01:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:02:00.751086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:02:00.752537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:02:00.831336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea886-a301-4508-a3d8-b9f4c716dcb9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:02:00.837960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:02:00.838578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:02:00.838857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:02:00.839131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:02:00.840874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:02:00.853010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:02:00.854683 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696bdeb6-7bd3-4b7f-b203-9ace63fa2d1b service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:02:00.865059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1222/2418] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:02:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:02:08.584944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=25 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:02:10.861308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=32 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:02:34.668200 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b0d7-8f9f-419a-b25f-8bea1f391c70 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:02:34.668738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b0d7-8f9f-419a-b25f-8bea1f391c70 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:02:34.668846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8b0d7-8f9f-419a-b25f-8bea1f391c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:02:34.688829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-88a8b0d7-8f9f-419a-b25f-8bea1f391c70 None None] MFA Rules not processed for user `272f5244f9724e00bd35dd09fedc6b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:02:34.690171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-88a8b0d7-8f9f-419a-b25f-8bea1f391c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:02:34.744897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1197/2419] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:02:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:02:44.738049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=25 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:08.359702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27510a7-a7ef-4ee3-aaf9-3486866c1ec9 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:08.360200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27510a7-a7ef-4ee3-aaf9-3486866c1ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:08.360310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27510a7-a7ef-4ee3-aaf9-3486866c1ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:08.393670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c27510a7-a7ef-4ee3-aaf9-3486866c1ec9 None None] MFA Rules not processed for user `bada6717873d408a95058051f2589a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:08.412361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c27510a7-a7ef-4ee3-aaf9-3486866c1ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:08.511976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1223/2420] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:13.583130 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:13.583694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:13.583694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:13.602836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] MFA Rules not processed for user `6a3c9d9bf3c8455aa93ad43b10bfd956`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:13.604032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:13.610918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1198/2421] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:14.080005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:14.080510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:14.080617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:14.100265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:14.101475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:14.174497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1224/2422] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:03:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:14.229640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:14.231406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:14.315043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a417c-afa5-4cf5-96d6-414a4be9700c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:14.326731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:14.327366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:14.327641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:14.327914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:14.336539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:14.338183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:14.346933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1199/2423] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:03:22.478550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:22.480131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:22.557053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce100b2-21ad-4cb1-9045-fe1e016f7f01 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:22.562514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:22.563270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:22.563756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:22.564195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] PATH_INFO: `/v3/users/67ec6b7d73494d9bb0294c21277407ce` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:22.565582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ec6b7d73494d9bb0294c21277407ce)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:22.568396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:22.630395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ec6b7d73494d9bb0294c21277407ce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.634141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:22.639373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ec6b7d73494d9bb0294c21277407ce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.640838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ec6b7d73494d9bb0294c21277407ce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.641715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ec6b7d73494d9bb0294c21277407ce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.650428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ec6b7d73494d9bb0294c21277407ce'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.670159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1225/2424] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:03:22 2022] DELETE /identity/v3/users/67ec6b7d73494d9bb0294c21277407ce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:22.837314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:22.848961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4cd1c-1d18-4b33-aa42-08dbbdc53ff7 service swift] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:22.919643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:22.920378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:22.920751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:22.921051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] PATH_INFO: `/v3/projects/29cd3d666355422e8ac5802fd07de461` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:22.922464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cd3d666355422e8ac5802fd07de461)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:22.924717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:22.980579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cd3d666355422e8ac5802fd07de461', 'request_id': 'req-a5fcee17-1ca0-42e5-a101-84c5071ebb51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.987680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cd3d666355422e8ac5802fd07de461', 'request_id': 'req-a5fcee17-1ca0-42e5-a101-84c5071ebb51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:22.997551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cd3d666355422e8ac5802fd07de461', 'request_id': 'req-a5fcee17-1ca0-42e5-a101-84c5071ebb51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.010807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] The token cache is being invalidate because project 29cd3d666355422e8ac5802fd07de461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:03:23.010807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.011528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.013391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1200/2425] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:03:22 2022] DELETE /identity/v3/projects/29cd3d666355422e8ac5802fd07de461 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:23.017051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.018564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:23.034395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc097f6-12b3-4025-b74d-da6a2d45f3ce admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.095357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.095985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.096262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.096534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] PATH_INFO: `/v3/users/33485026d0ed4151abe151d0cdbcd177` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.097669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33485026d0ed4151abe151d0cdbcd177)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.100196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.146555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33485026d0ed4151abe151d0cdbcd177'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.155306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33485026d0ed4151abe151d0cdbcd177'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.155935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33485026d0ed4151abe151d0cdbcd177'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.157017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33485026d0ed4151abe151d0cdbcd177'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.165232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-6a4f2722-c0a8-431f-9c84-8db70720f35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33485026d0ed4151abe151d0cdbcd177'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.186814 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1226/2426] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:03:23 2022] DELETE /identity/v3/users/33485026d0ed4151abe151d0cdbcd177 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:23.375405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.376832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:23.390225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcee17-1ca0-42e5-a101-84c5071ebb51 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.401256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:23.455169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.455800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.456075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.456344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] PATH_INFO: `/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.457643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071020b20eab4c8e99e1a89e8e4d27f7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.459603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.509162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071020b20eab4c8e99e1a89e8e4d27f7', 'request_id': 'req-402774bf-9737-4965-adbd-99e8d89482e1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.515381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071020b20eab4c8e99e1a89e8e4d27f7', 'request_id': 'req-402774bf-9737-4965-adbd-99e8d89482e1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.522853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071020b20eab4c8e99e1a89e8e4d27f7', 'request_id': 'req-402774bf-9737-4965-adbd-99e8d89482e1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.532862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] The token cache is being invalidate because project 071020b20eab4c8e99e1a89e8e4d27f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:03:23.533147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.533782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-402774bf-9737-4965-adbd-99e8d89482e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:23.535890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1201/2427] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:03:23 2022] DELETE /identity/v3/projects/071020b20eab4c8e99e1a89e8e4d27f7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:23.540917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.541370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.541462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.561436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:23.562670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:23.605603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:23.622742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1227/2428] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:23.630493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.630872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.631098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.651578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:23.652748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:23.662762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1202/2429] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:23.667154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.679169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5eba1-22a8-422f-99c5-19587379a7bf None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.683822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.684457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.684780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.685055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.685704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:03:23.686038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.687664 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.700910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1228/2430] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:03:23.705331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.715189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76c719-f5b8-490b-8d6a-72416b7f3f88 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.720080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.720677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.720954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.721224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.722029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.724046 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.752189 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1203/2431] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:03:23 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:03:23.756720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.766998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35650fc7-408e-41fe-8cda-402a7d77c26e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.822196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.823048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.823322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.823586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.824199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.826134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.846932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1229/2432] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:03:23 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:03:23.851578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.861678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7d9f-d92c-4bd2-be7e-8f88d5190581 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.866248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.866834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.867108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.867381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.868020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.869603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.879283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1204/2433] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:23.883005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.893025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b1e43-b437-4e85-a238-f96ae6fd5b2e admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.898355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.898952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.899226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.899494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.900144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.901801 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15054f57-8dcf-4f86-9282-f5dcf39104e3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.913229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1230/2434] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:23.917397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:23.927366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c458067-4bbe-4080-9e5d-10ae5223f41a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:23.931968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:23.932604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.932879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.933149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] PATH_INFO: `/v3/projects/f7a64886550f4140bf90a77070f769e0/users/f07e6a2e509e4aba865fcd6e935afa2a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.947698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a64886550f4140bf90a77070f769e0, user_id=f07e6a2e509e4aba865fcd6e935afa2a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:23.950361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:23.964029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1205/2435] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:03:23 2022] PUT /identity/v3/projects/f7a64886550f4140bf90a77070f769e0/users/f07e6a2e509e4aba865fcd6e935afa2a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:03:23.969071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72aca7-03d4-4a5f-9f1d-3a779ef6cd65 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:23.969355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72aca7-03d4-4a5f-9f1d-3a779ef6cd65 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:23.969628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72aca7-03d4-4a5f-9f1d-3a779ef6cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:23.988869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-be72aca7-03d4-4a5f-9f1d-3a779ef6cd65 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:23.990075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-be72aca7-03d4-4a5f-9f1d-3a779ef6cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:24.054904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1231/2436] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:24.065083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:24.067813 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:24.120990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:24.137939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1442c8-22b9-4541-8f2d-88a85790be25 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:24.141892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:24.142509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:24.142788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:24.143059 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:24.153288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:24.154713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:24.168002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1206/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:03:30.470651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:30.471141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:30.471250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:30.509035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] MFA Rules not processed for user `f07e6a2e509e4aba865fcd6e935afa2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:30.510607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:30.574625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:30.602060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1232/2438] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:30.606250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:30.619659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff9bdd-23ba-4e16-bfd9-8e50ad166c32 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:30.636351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:30.636977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:30.637253 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:30.637518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:30.638190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:30.640158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:30.713172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1207/2439] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:03:30 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:03:30.713910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:30.728215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b8494-7b24-436d-bd4f-f1fd02892a55 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:30.821932 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:30.822913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:30.823233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:30.824640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:30.825155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:30.827531 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:30.858765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1233/2440] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:03:30 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:03:30.865608 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:30.896108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40975fd8-d0e7-4ae7-8937-3f6ba25db716 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:30.909267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:30.910803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:30.911262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:30.911929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:30.912937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:30.917368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:30.938044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1208/2441] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:30.942278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:30.955967 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ead9-c65b-4755-a8a7-3864125fd9cb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:30.960780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:30.961457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:30.961749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:30.962033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] PATH_INFO: `/v3/projects/01408158bb0345bf9b8f5ff547c84bcb/users/985634f35e8d42a78ca53eac3924fb35/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:30.979230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01408158bb0345bf9b8f5ff547c84bcb, user_id=985634f35e8d42a78ca53eac3924fb35, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:30.981922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:31.000156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1234/2442] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:03:30 2022] PUT /identity/v3/projects/01408158bb0345bf9b8f5ff547c84bcb/users/985634f35e8d42a78ca53eac3924fb35/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:03:31.001417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:31.013347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03566c2-a833-471b-97f2-49d6b0338458 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:31.129384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:31.130068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:31.145341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:31.145697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:31.146494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:31.148150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a12ee-467c-4a57-9dd1-a44f48674475 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:31.176015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1209/2443] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:31.180462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:31.196136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662145-5c9e-4adc-adae-659615a468c9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:31.214118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:31.214914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:31.215111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:31.215416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] PATH_INFO: `/v3/projects/01408158bb0345bf9b8f5ff547c84bcb/users/985634f35e8d42a78ca53eac3924fb35/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:31.218377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01408158bb0345bf9b8f5ff547c84bcb, user_id=985634f35e8d42a78ca53eac3924fb35, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:31.223963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:31.239991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1235/2444] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:03:31 2022] PUT /identity/v3/projects/01408158bb0345bf9b8f5ff547c84bcb/users/985634f35e8d42a78ca53eac3924fb35/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:36.601445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c0503-f989-4c78-a178-4ae77e9f0090 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:36.602063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c0503-f989-4c78-a178-4ae77e9f0090 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:36.602355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c0503-f989-4c78-a178-4ae77e9f0090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:36.635421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-767c0503-f989-4c78-a178-4ae77e9f0090 None None] MFA Rules not processed for user `985634f35e8d42a78ca53eac3924fb35`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:36.639245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-767c0503-f989-4c78-a178-4ae77e9f0090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:36.754680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1210/2445] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:36.768662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:36.770483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:36.864794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afe702-76fd-48ec-845c-1e2e4af1f1e5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:36.870459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:36.871126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:36.871405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:36.871678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:36.908516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:36.935532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:36.937146 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:36.946408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1236/2446] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:03:37.967436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:37.967958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:37.967958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:37.988399 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] MFA Rules not processed for user `f07e6a2e509e4aba865fcd6e935afa2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:37.989634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:37.996413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1211/2447] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:44.110999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.112374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:44.192167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d351c6b-70f2-4f9f-8b24-01f28a4f34a5 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.196887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.197838 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.198124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.198405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] PATH_INFO: `/v3/users/272f5244f9724e00bd35dd09fedc6b67` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.199542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=272f5244f9724e00bd35dd09fedc6b67)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.201477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.249165 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272f5244f9724e00bd35dd09fedc6b67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.257071 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272f5244f9724e00bd35dd09fedc6b67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.257904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272f5244f9724e00bd35dd09fedc6b67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.258544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272f5244f9724e00bd35dd09fedc6b67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.265695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-99299ae1-8006-450e-be81-33c5899f6c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272f5244f9724e00bd35dd09fedc6b67'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.283420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1237/2448] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:03:44 2022] DELETE /identity/v3/users/272f5244f9724e00bd35dd09fedc6b67 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:44.460797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.474428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbd1ce-9141-4e21-8c03-e2614b81a1ac None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.502675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:44.541249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.541926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.542209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.542484 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] PATH_INFO: `/v3/projects/3414a292093541209ace108ad8bb6be4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.543626 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3414a292093541209ace108ad8bb6be4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.545587 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.595844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3414a292093541209ace108ad8bb6be4', 'request_id': 'req-9f49a975-3de2-406a-b9f3-db63fea1a45f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.603925 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3414a292093541209ace108ad8bb6be4', 'request_id': 'req-9f49a975-3de2-406a-b9f3-db63fea1a45f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.614413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3414a292093541209ace108ad8bb6be4', 'request_id': 'req-9f49a975-3de2-406a-b9f3-db63fea1a45f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.624872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] The token cache is being invalidate because project 3414a292093541209ace108ad8bb6be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:03:44.625160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.625806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f49a975-3de2-406a-b9f3-db63fea1a45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:03:44.627417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1212/2449] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:03:44 2022] DELETE /identity/v3/projects/3414a292093541209ace108ad8bb6be4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:44.632410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.632691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.632962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.652907 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:44.654062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:44.707863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:44.721212 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1238/2450] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:44.730265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.730924 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.731479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.755490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:44.756705 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:44.764659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1213/2451] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:44.768991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.778669 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5417e6c-6445-49f8-a9f5-3101f44824dd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.783455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.784054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.784332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.784659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.785307 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:03:44.785612 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.787269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.801476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1239/2452] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:03:44.805674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.815452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c184a8-c846-4779-a310-31e8c9e20477 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.820830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.821430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.821709 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.822014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.822919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.824835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.855873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1214/2453] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:03:44 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:03:44.857114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.870992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994a552-1683-4bbf-8f12-42b716085d04 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.933120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.933648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.933960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.934236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.934899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.936847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.959051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1240/2454] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:03:44 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:03:44.963271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:44.973505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7869ee1-2466-4d51-a98e-baad07baad0e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:44.978611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:44.979240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:44.979521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:44.979792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:44.980485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:44.982318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:44.993554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1215/2455] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:44.997824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:45.009290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24f61-0122-44b0-bac2-4155f10e31ab admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:45.014074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:45.014674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:45.014976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:45.015253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:45.015934 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:45.017553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2973a0-c591-4d41-a75b-db72d3731dd2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:45.030477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1241/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:03:45.035378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:45.048458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fb6a5-0a26-4a4d-8bbe-edb402715a7c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:45.053811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:45.054465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:45.054745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:45.055017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] PATH_INFO: `/v3/projects/be59f6fed9f645d5bd7478535a2fcb02/users/a4828b30dda248758b4161eafe8668c5/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:45.069472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be59f6fed9f645d5bd7478535a2fcb02, user_id=a4828b30dda248758b4161eafe8668c5, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:45.072158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:45.085171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1216/2457] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:03:45 2022] PUT /identity/v3/projects/be59f6fed9f645d5bd7478535a2fcb02/users/a4828b30dda248758b4161eafe8668c5/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:03:45.090761 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fe81-9a44-4cf5-887f-aa0c751177e6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:45.091045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fe81-9a44-4cf5-887f-aa0c751177e6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:45.091319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2fe81-9a44-4cf5-887f-aa0c751177e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:45.111869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-75b2fe81-9a44-4cf5-887f-aa0c751177e6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:45.113874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-75b2fe81-9a44-4cf5-887f-aa0c751177e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:45.172141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1242/2458] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:45.182440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:45.184198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:45.247840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ed80b-3120-4cf0-8fba-2742524f80d1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:45.251624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:45.252230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:45.252505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:45.252773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:45.258673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:45.263618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:45.266041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:45.275854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1217/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:03:49.370690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69eb41-8c68-4e6f-b223-405ac09bf1da None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:49.371182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69eb41-8c68-4e6f-b223-405ac09bf1da None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:49.371276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69eb41-8c68-4e6f-b223-405ac09bf1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:49.410438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0a69eb41-8c68-4e6f-b223-405ac09bf1da None None] MFA Rules not processed for user `a4828b30dda248758b4161eafe8668c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:49.411706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0a69eb41-8c68-4e6f-b223-405ac09bf1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:49.468053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:03:49.489986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1243/2460] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:49.507238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:49.508691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:49.599969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2ef2-26f4-4ed7-8ead-48d61c4e306d service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:49.606243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:49.607087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:49.607372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:49.607713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:49.617367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:49.619035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:49.629858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1218/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:03:50.951269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:50.951735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:50.951849 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:50.975593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] MFA Rules not processed for user `a4828b30dda248758b4161eafe8668c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:03:50.976811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:50.983681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1244/2462] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:03:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:03:51.613794 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:51.615577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:51.629441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec9c4-6096-4e5a-b381-b9b934155bd7 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:51.633636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:51.634482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:51.634918 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:51.635362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:51.637498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:51.651845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:51.653281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:51.666133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1219/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:03:52.812156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:03:52.826696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552b6ea-0e7d-48b9-b24b-175ec5e93efb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:03:52.832220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:03:52.833089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:03:52.834026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:03:52.834204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:03:52.837595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:03:52.859053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:03:52.860836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:03:52.875791 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1245/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:01.661552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=43 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:02.870689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=14 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:05.064437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:05.066012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:05.079688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156c34c-217a-41ed-8e9b-b6b0b26f972b service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:05.085110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:05.085695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:05.086017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:05.086284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:05.087933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:05.146877 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:05.148390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:05.157539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1220/2465] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:11.869867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:11.878874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01882eb1-600f-4adb-87e7-eb0c8c6fdf46 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:11.882510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:11.883106 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:11.883386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:11.883655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:11.885383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:11.949226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:11.950705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:11.960749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1246/2466] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:13.897269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:13.910445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab16f8-8a30-44a5-895f-fc2aae78d5f3 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:13.914184 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:13.914873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:13.915216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:13.915552 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:13.917376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:13.927334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:13.928840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54befb99-8898-41d7-b504-1121d3a8987d service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:13.938223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1221/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:21.956578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=30 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:23.934392 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=34 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:30.634767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:30.652707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db7fa6-708d-4c8d-bf2f-8591a4cb9b55 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:30.661521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:30.662175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:30.662456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:30.662727 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:30.665251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:30.681256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:30.683251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:30.706334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1247/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:34.762446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d3bac-bd45-4de0-afe6-0db74587b76f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:34.763400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d3bac-bd45-4de0-afe6-0db74587b76f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:34.763676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d3bac-bd45-4de0-afe6-0db74587b76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:34.783229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4c9d3bac-bd45-4de0-afe6-0db74587b76f None None] MFA Rules not processed for user `6a3c9d9bf3c8455aa93ad43b10bfd956`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:34.784423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4c9d3bac-bd45-4de0-afe6-0db74587b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:34.840553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1222/2469] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:04:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:35.166152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:35.175593 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71cf77-3ae9-4e35-a83e-cda8c22bcf6a service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:35.179460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:35.180085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:35.180368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:35.180637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:35.182362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:35.192598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:35.194060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6d18-e789-4380-bf4f-0d8a54df74eb service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:35.202502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1248/2470] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:35.634730 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646c7b-cd8a-4151-94ca-72a6a56a0064 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:35.635217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646c7b-cd8a-4151-94ca-72a6a56a0064 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:35.635675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646c7b-cd8a-4151-94ca-72a6a56a0064 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:35.637182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1223/2471] 10.222.0.15 () {54 vars in 961 bytes} [Tue Sep 13 08:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:04:35.642926 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:35.643459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:35.644000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:35.665076 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] MFA Rules not processed for user `e6fd2b2ee55f419195b935ba13743dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:35.665965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:35.731732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1249/2472] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Sep 13 08:04:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:35.739687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c646c7b-cd8a-4151-94ca-72a6a56a0064 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:35.752878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c646c7b-cd8a-4151-94ca-72a6a56a0064 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:35.757838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:35.758490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:35.758711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:35.758980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:35.760682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:35.773856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:35.775499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:35.786308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1224/2473] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:04:40.261594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:40.271270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbcb79-efa9-4d55-ad1d-2edfd94efdfe None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:40.274785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:40.275373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:40.275645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:40.275910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:40.277585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:40.289119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:40.290570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:40.300538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1250/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:43.553551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:43.573107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3fa50-e4bd-44a7-8b53-9b747964e3c9 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:43.581178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:43.581980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:43.582333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:43.582680 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:43.584717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:43.598166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:43.600070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:43.612294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1225/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:43.870682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:43.872310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:43.885665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ec4fe-2dd1-41aa-aed6-bd2a6553e4a3 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:43.891285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:43.892057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:43.892399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:43.892737 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] PATH_INFO: `/v3/users/bada6717873d408a95058051f2589a89` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:43.893973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bada6717873d408a95058051f2589a89)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:43.895982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:43.941433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bada6717873d408a95058051f2589a89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:43.948283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bada6717873d408a95058051f2589a89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:43.948905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bada6717873d408a95058051f2589a89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:43.949501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bada6717873d408a95058051f2589a89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:43.956595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c8ed8db5-cfaf-4bd8-b4f6-0fdc686a1d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bada6717873d408a95058051f2589a89'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:43.974075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1251/2476] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:04:43 2022] DELETE /identity/v3/users/bada6717873d408a95058051f2589a89 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:04:44.142756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.155356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6b83d-faa0-4855-8a0f-0c24565b99a4 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.212800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.213432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.213713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.214016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] PATH_INFO: `/v3/projects/673e7e0a20014808a399969e1d5e0d41` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.215137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] RBAC: Authorizing `identity:delete_project(project_id=673e7e0a20014808a399969e1d5e0d41)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.217054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.263820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673e7e0a20014808a399969e1d5e0d41', 'request_id': 'req-8222502a-9a8f-4f17-9ce1-c53403ee849c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:44.270280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673e7e0a20014808a399969e1d5e0d41', 'request_id': 'req-8222502a-9a8f-4f17-9ce1-c53403ee849c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:44.277581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673e7e0a20014808a399969e1d5e0d41', 'request_id': 'req-8222502a-9a8f-4f17-9ce1-c53403ee849c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:44.288464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] The token cache is being invalidate because project 673e7e0a20014808a399969e1d5e0d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:04:44.288753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:44.289385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-8222502a-9a8f-4f17-9ce1-c53403ee849c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:04:44.291439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1226/2477] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:04:44 2022] DELETE /identity/v3/projects/673e7e0a20014808a399969e1d5e0d41 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:04:44.296232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.296518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.296789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.316443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:44.317686 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:44.341568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:44.378508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1252/2478] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:44.386355 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.386667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.386939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.406580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:44.408378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:44.416254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1227/2479] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:44.420207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.429626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c746414-1843-4dc4-aa23-5099f79eda7c None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.434236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.434834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.435112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.435383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.436023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:04:44.436321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.437952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.449676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1253/2480] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 08:04:44.454788 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.459202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:44.465178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a799e-d728-4f86-a29b-dbe62447e611 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.472887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.477075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.477075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.477075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.477075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.477498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.504506 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1228/2481] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:04:44.508897 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.519283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af386e8a-74c0-403e-8387-482f61cb1906 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.575675 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.576287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.576565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.576837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.577473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.579946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.600885 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1254/2482] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:04:44.605255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.615607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9ff79-817a-4b10-b6ba-750befdc1c0a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.620340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.620933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.621207 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.621474 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.622140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.623724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.633337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1229/2483] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:04:44.637168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.647490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1af4a-2483-410f-96fa-99d84719b331 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.652095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.652679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.652955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.653225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.653894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.655474 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09280ea-cd3f-4d17-8fa4-3e82e53d91a7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.666634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1255/2484] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:04:44.670617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.682116 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69dbe4-d742-48da-877c-55e9b03c6149 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.686782 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.687423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.687698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.687964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] PATH_INFO: `/v3/projects/86a21dc1423a4f2a9fb835b4a7f2b97c/users/0ddae6146df34d6fa4529a0bcc1ddd62/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.702930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a21dc1423a4f2a9fb835b4a7f2b97c, user_id=0ddae6146df34d6fa4529a0bcc1ddd62, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.705662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.723983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1230/2485] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:04:44 2022] PUT /identity/v3/projects/86a21dc1423a4f2a9fb835b4a7f2b97c/users/0ddae6146df34d6fa4529a0bcc1ddd62/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:04:44.735139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.735423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.736324 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.764023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] MFA Rules not processed for user `0ddae6146df34d6fa4529a0bcc1ddd62`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:44.765217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:44.829773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1256/2486] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:44.834384 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.904618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fb0e6-2559-4836-92c6-855eaaa03a12 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.909887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:44.910520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:44.910792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:44.911057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:44.911701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:44.913592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:44.940157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1231/2487] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:04:44.944217 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:44.955495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e7828-ed61-4e9b-8ae4-14852f42b965 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:44.975870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:45.014097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.014699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.014976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.015246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.015887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.017813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.038762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1257/2488] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:04:44 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:04:45.042822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:45.053693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364aa6-f882-479e-8a52-ea65399910e2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:45.058383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.058984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.059261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.059533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.060171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.061786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.072656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1232/2489] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:04:45.076665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:45.087015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd3eef-c668-459c-ac8f-029ee400168b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:45.091604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.092243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.092525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.092795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] PATH_INFO: `/v3/projects/f56a534daf584d5a96fa29dc962f3aaa/users/05c49b60c8f846599fcfa18050b660b7/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.107784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56a534daf584d5a96fa29dc962f3aaa, user_id=05c49b60c8f846599fcfa18050b660b7, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.110517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.123162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1258/2490] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:04:45 2022] PUT /identity/v3/projects/f56a534daf584d5a96fa29dc962f3aaa/users/05c49b60c8f846599fcfa18050b660b7/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:04:45.126962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:45.136882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093d0df-6e92-4266-ba7f-1adb3005f6e9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:45.192797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.193301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.193580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.193954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.194593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.196233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86715c4a-b241-41b3-ae38-fdceb3e62d21 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.206464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1233/2491] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:04:45.210645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:45.221385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd8db5-d19e-4cd6-aad8-c0d9ca744ef1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:45.225972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.226618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.226896 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.227167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] PATH_INFO: `/v3/projects/f56a534daf584d5a96fa29dc962f3aaa/users/05c49b60c8f846599fcfa18050b660b7/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.229064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56a534daf584d5a96fa29dc962f3aaa, user_id=05c49b60c8f846599fcfa18050b660b7, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.231721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.241753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1259/2492] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:04:45 2022] PUT /identity/v3/projects/f56a534daf584d5a96fa29dc962f3aaa/users/05c49b60c8f846599fcfa18050b660b7/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:04:45.253177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.253462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.253767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.281143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] MFA Rules not processed for user `05c49b60c8f846599fcfa18050b660b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:04:45.282944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:45.286874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:04:45.352595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1234/2493] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:04:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:04:45.363514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:45.365206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:45.424608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9134e1-4b23-4edb-a15f-9dbe86b16250 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:45.428440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:45.429061 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:45.429336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:45.429605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:45.489516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:45.490947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:45.499865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1260/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:50.126544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:04:50.127973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:04:50.140686 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955253d2-6e68-4121-be30-357e84f5a0e7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:04:50.145224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:04:50.145869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:04:50.146149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:04:50.146427 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:04:50.155724 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:04:50.157156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c45fb-4681-43aa-afd6-2e0bfb40a235 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:04:50.166558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1235/2495] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:04:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:04:55.495773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:00.162455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=33 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:11.923454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:11.985225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab6e20-f50d-48a9-b9b8-42ddee4e6b83 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:11.990058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:11.990680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:11.990956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:11.991226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] PATH_INFO: `/v3/users/0ddae6146df34d6fa4529a0bcc1ddd62` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:11.992473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ddae6146df34d6fa4529a0bcc1ddd62)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:11.994450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:12.040281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ddae6146df34d6fa4529a0bcc1ddd62'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.046618 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ddae6146df34d6fa4529a0bcc1ddd62'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.047741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ddae6146df34d6fa4529a0bcc1ddd62'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.048345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ddae6146df34d6fa4529a0bcc1ddd62'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.055232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ddae6146df34d6fa4529a0bcc1ddd62'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.073225 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1261/2496] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:11 2022] DELETE /identity/v3/users/0ddae6146df34d6fa4529a0bcc1ddd62 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:12.077692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:12.078190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:12.078403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:12.102883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:12.104604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:12.170351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1236/2497] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:12.180700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:12.182158 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:12.242010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f0d15-a764-4b4c-83d0-61ee57b19f89 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:12.245546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:12.246180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:12.246454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:12.246722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:12.256122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:12.257536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:12.266765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1262/2498] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:12.531543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:12.533075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:12.548383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec7c36-a4ad-4310-b7c0-e0e0a13f42eb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:12.559687 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:12.560391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:12.560669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:12.560965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] PATH_INFO: `/v3/projects/86a21dc1423a4f2a9fb835b4a7f2b97c` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:12.562171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a21dc1423a4f2a9fb835b4a7f2b97c)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:12.564599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:12.611577 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a21dc1423a4f2a9fb835b4a7f2b97c', 'request_id': 'req-6b206852-f64a-4c2b-b0e9-40c985f8fdef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.617829 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a21dc1423a4f2a9fb835b4a7f2b97c', 'request_id': 'req-6b206852-f64a-4c2b-b0e9-40c985f8fdef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.626894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a21dc1423a4f2a9fb835b4a7f2b97c', 'request_id': 'req-6b206852-f64a-4c2b-b0e9-40c985f8fdef'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.636973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] The token cache is being invalidate because project 86a21dc1423a4f2a9fb835b4a7f2b97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:12.637259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.637906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.639376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1237/2499] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:12 2022] DELETE /identity/v3/projects/86a21dc1423a4f2a9fb835b4a7f2b97c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:12.644032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:12.653584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d839417-e1b9-4d1b-806d-33e3406d30b4 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:12.671161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:12.711816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:12.712441 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:12.712715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:12.712984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] PATH_INFO: `/v3/users/05c49b60c8f846599fcfa18050b660b7` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:12.714148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c49b60c8f846599fcfa18050b660b7)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:12.716057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:12.760151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c49b60c8f846599fcfa18050b660b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.767575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c49b60c8f846599fcfa18050b660b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.768276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c49b60c8f846599fcfa18050b660b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.768891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c49b60c8f846599fcfa18050b660b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.776740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-118099fa-fe2a-47c0-a254-bab5f97dd80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c49b60c8f846599fcfa18050b660b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:12.794512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1263/2500] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:12 2022] DELETE /identity/v3/users/05c49b60c8f846599fcfa18050b660b7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:13.073485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.075063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:13.090748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b206852-f64a-4c2b-b0e9-40c985f8fdef admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.148105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.148737 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.149016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.149286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] PATH_INFO: `/v3/projects/f56a534daf584d5a96fa29dc962f3aaa` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.150437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56a534daf584d5a96fa29dc962f3aaa)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.152368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.182318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:13.200540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56a534daf584d5a96fa29dc962f3aaa', 'request_id': 'req-573880bb-3799-48ae-96a2-3601154fe82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:13.206795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56a534daf584d5a96fa29dc962f3aaa', 'request_id': 'req-573880bb-3799-48ae-96a2-3601154fe82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:13.214366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56a534daf584d5a96fa29dc962f3aaa', 'request_id': 'req-573880bb-3799-48ae-96a2-3601154fe82b'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:13.225488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] The token cache is being invalidate because project f56a534daf584d5a96fa29dc962f3aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:13.225830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:13.226451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-573880bb-3799-48ae-96a2-3601154fe82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:13.227962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1238/2501] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:13 2022] DELETE /identity/v3/projects/f56a534daf584d5a96fa29dc962f3aaa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:13.232825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.233276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.233394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.253214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:13.254438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:13.313283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1264/2502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:13.322727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.323132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.323471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.343161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:13.344969 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:13.353143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1239/2503] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:13.357350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.367397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a31be-cabb-4239-8086-8c7663200a56 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.372315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.372923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.373200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.373468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.374155 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:05:13.374456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.376624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.388786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1265/2504] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:05:13.393655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.403175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97127527-5a89-4be7-9a00-9a469128b213 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.407946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.408671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.408849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.409073 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.409718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.412118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.440583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1240/2505] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:13.444873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.455595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe64460-c636-40ee-bfa6-7d70bb3d1609 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.481863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:13.514850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.515467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.515787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.516560 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.517196 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.519139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.541046 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1266/2506] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:13.545301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.556094 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701a7d-bc13-44ff-8520-d4b61509bb0e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.560632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.561232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.561509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.561806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.562472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.564653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.575000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1241/2507] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:13.578766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.589286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499056a-6ed2-4337-96fd-6676c49e4f7f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.593975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.594681 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.594965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.595235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.595880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.597471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e8632-f896-4e89-83c6-a1a3dc241b61 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.607764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1267/2508] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:13.612526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.623191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d0d13-24be-4b40-9e5d-95c5fa5554b0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.629187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.629946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.630230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.630505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] PATH_INFO: `/v3/projects/1f613f6d523a4b3ebe35fbe33d552943/users/1c04333f2d9a49b29fb1ed76c1941c13/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.645282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f613f6d523a4b3ebe35fbe33d552943, user_id=1c04333f2d9a49b29fb1ed76c1941c13, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.648006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.660965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1242/2509] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:13 2022] PUT /identity/v3/projects/1f613f6d523a4b3ebe35fbe33d552943/users/1c04333f2d9a49b29fb1ed76c1941c13/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:13.671479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.671763 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.672559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.704149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] MFA Rules not processed for user `1c04333f2d9a49b29fb1ed76c1941c13`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:13.705465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:13.774017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1268/2510] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:13.778733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.841710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69bfbe-37ea-4a4e-9b3a-a39a3799de31 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.847005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.847612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.847891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.848165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.848812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.850763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.879634 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1243/2511] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:13.883673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.895236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ba7e7-0638-46d2-a4b8-89bec42e44b3 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.952585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.954200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.954200 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.954389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:13.954989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:13.956980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:13.978448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1269/2512] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:13 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:13.982657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:13.993503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52236478-fba6-4d2c-a407-3e2295ef5783 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:13.998329 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:13.998927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:13.999206 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:13.999477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.000657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.002274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.013219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1244/2513] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:14.017877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.027815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cec5a2-3f1f-4e33-aaf2-42ad082a1718 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.032304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.032944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.033219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.033491 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] PATH_INFO: `/v3/projects/9805be66631646f39d6c395b1e2c8b19/users/c3df80dfffab4c99bde08a40e8852cfe/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.048098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9805be66631646f39d6c395b1e2c8b19, user_id=c3df80dfffab4c99bde08a40e8852cfe, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.050772 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.064465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1270/2514] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:14 2022] PUT /identity/v3/projects/9805be66631646f39d6c395b1e2c8b19/users/c3df80dfffab4c99bde08a40e8852cfe/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:14.068406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.079378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa8cd-9d67-437f-8780-83ea6475bd04 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.098938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:14.138593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.139309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.139651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.140012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.141225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.142839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee005c9c-c642-41fc-8a88-24ad68aa6efe admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.152961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1245/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:14.157190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.162622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:14.169891 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50668d7-0c52-4219-bb49-96461506a745 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.175584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.176271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.176547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.176816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] PATH_INFO: `/v3/projects/9805be66631646f39d6c395b1e2c8b19/users/c3df80dfffab4c99bde08a40e8852cfe/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.179747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=9805be66631646f39d6c395b1e2c8b19, user_id=c3df80dfffab4c99bde08a40e8852cfe, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.182957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96058a28-8cd0-41fe-8a43-55aaf8525dde admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.194454 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1271/2516] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:14 2022] PUT /identity/v3/projects/9805be66631646f39d6c395b1e2c8b19/users/c3df80dfffab4c99bde08a40e8852cfe/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:14.206658 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.206943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.207367 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.235951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] MFA Rules not processed for user `c3df80dfffab4c99bde08a40e8852cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:14.237166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:14.268795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.269301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.269301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.288424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:14.289606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:14.310977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1246/2517] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:14.322970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.324477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:14.346871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1272/2518] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:05:14.386706 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccc475-0e56-40dc-a468-7310d1c61de1 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.390614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.391220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.391495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.391762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.396596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.398223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:14.401340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.402798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.412446 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1247/2519] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:14.493816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a62709-13ce-4bea-b11c-595cdf0a828b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.499211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.499831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.500803 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.501128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.510140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.512580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c0d3a-b6d6-49a9-8203-72a6000472ae service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:14.531609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1273/2520] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 08:05:14.865455 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:14.935602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:14.953458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfee115-8dc0-48af-bd59-155b572ee5b7 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:14.958135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:14.958836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:14.959181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:14.959547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] PATH_INFO: `/v3/users/1c04333f2d9a49b29fb1ed76c1941c13` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:14.960756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c04333f2d9a49b29fb1ed76c1941c13)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:14.963176 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:15.014516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c04333f2d9a49b29fb1ed76c1941c13'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.023316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c04333f2d9a49b29fb1ed76c1941c13'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.023848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c04333f2d9a49b29fb1ed76c1941c13'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.024630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c04333f2d9a49b29fb1ed76c1941c13'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.033204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c04333f2d9a49b29fb1ed76c1941c13'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.056538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1248/2521] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:14 2022] DELETE /identity/v3/users/1c04333f2d9a49b29fb1ed76c1941c13 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:15.061124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:15.061411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:15.061692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:15.084621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:15.086086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:15.162406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1274/2522] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:15.173636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:15.175262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:15.241965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a2a2c-da02-4343-bb22-65c8b504526a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:15.246291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:15.247535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:15.248357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:15.248357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:15.259805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:15.262720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:15.273483 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1249/2523] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:15.590210 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:15.591769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:15.605588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90519ab-e22c-4f79-b5dd-10160e77de81 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:15.611648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:15.612466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:15.612802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:15.613168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] PATH_INFO: `/v3/projects/1f613f6d523a4b3ebe35fbe33d552943` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:15.614510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f613f6d523a4b3ebe35fbe33d552943)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:15.616957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:15.645629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:15.683784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f613f6d523a4b3ebe35fbe33d552943', 'request_id': 'req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.693291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f613f6d523a4b3ebe35fbe33d552943', 'request_id': 'req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.705839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f613f6d523a4b3ebe35fbe33d552943', 'request_id': 'req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.722661 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] The token cache is being invalidate because project 1f613f6d523a4b3ebe35fbe33d552943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:15.723249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.724025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.730118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1275/2524] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:15 2022] DELETE /identity/v3/projects/1f613f6d523a4b3ebe35fbe33d552943 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:15.730617 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:15.741479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb8c26-14d8-4f36-9267-c26c4e33f30d service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:15.846030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:15.846744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:15.847151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:15.847522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] PATH_INFO: `/v3/users/c3df80dfffab4c99bde08a40e8852cfe` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:15.849014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3df80dfffab4c99bde08a40e8852cfe)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:15.851581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:15.872611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:15.965599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3df80dfffab4c99bde08a40e8852cfe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.976033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3df80dfffab4c99bde08a40e8852cfe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.976615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3df80dfffab4c99bde08a40e8852cfe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.978574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3df80dfffab4c99bde08a40e8852cfe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:15.988825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-7f2b1927-13c8-4311-980f-0c72b4cff19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3df80dfffab4c99bde08a40e8852cfe'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.042961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1250/2525] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:15 2022] DELETE /identity/v3/users/c3df80dfffab4c99bde08a40e8852cfe => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:16.371731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:16.389824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:16.414385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eb010-16cb-4e1c-9d1e-1a5df6fd55a2 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:16.485544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:16.486503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.486780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.487385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] PATH_INFO: `/v3/projects/9805be66631646f39d6c395b1e2c8b19` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.488916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9805be66631646f39d6c395b1e2c8b19)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:16.490958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:16.552755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9805be66631646f39d6c395b1e2c8b19', 'request_id': 'req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.560277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9805be66631646f39d6c395b1e2c8b19', 'request_id': 'req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.571156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9805be66631646f39d6c395b1e2c8b19', 'request_id': 'req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.584289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] The token cache is being invalidate because project 9805be66631646f39d6c395b1e2c8b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:16.584582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.585311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df77efb2-5d30-4aa4-b6f5-05f3ea7724b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:16.586966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1276/2526] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:16 2022] DELETE /identity/v3/projects/9805be66631646f39d6c395b1e2c8b19 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:16.592264 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.592791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.592903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.615803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:16.617057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:16.691087 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1251/2527] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:16.698767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.699291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.699291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.721277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:16.722550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:16.733775 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1277/2528] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:16.737998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:16.757458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773eb5a-3ada-4a2a-a2d4-90cd0173988d None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:16.763641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:16.764222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.764505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.764777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.766031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:05:16.766341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:16.769323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:16.784821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1252/2529] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:05:16.789415 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:16.799575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f142237-e340-42cc-901e-55d5e6823513 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:16.807248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:16.808879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.809459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.809999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.811199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:16.813758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:16.848994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1278/2530] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:16 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:16.853518 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:16.864711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a11b1-a3a6-49b6-9f6f-cb18bb6aefbb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:16.940488 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:16.946603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:16.946825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:16.947107 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:16.948032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:16.950226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:16.983247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1253/2531] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:16 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:16.984946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:17.003601 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efef02-1d86-4cca-8594-9dc0ff95eb5c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:17.008620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:17.009623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:17.010084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:17.012276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:17.012991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:17.015086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:17.017819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:17.032150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1279/2532] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:17.035708 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:17.052148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718bfe1-533f-430f-9587-140e5d5959b5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:17.069035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:17.069668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:17.069998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:17.070486 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:17.072612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:17.073018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f031cd5-c9fd-45a8-a44c-36eb63c86bfb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:17.075535 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:17.103521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:17.111848 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1254/2533] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:17.115946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325545cb-95eb-41ac-8fd8-58b16809b72f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:17.128078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:17.129035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:17.129689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:17.130215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] PATH_INFO: `/v3/projects/0a6bd12fa7d04b8e866bdaffc42493f4/users/7389d11dbd964b87ae6ad774cbcfe16f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:17.152052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6bd12fa7d04b8e866bdaffc42493f4, user_id=7389d11dbd964b87ae6ad774cbcfe16f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:17.154771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:17.170351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1280/2534] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:17 2022] PUT /identity/v3/projects/0a6bd12fa7d04b8e866bdaffc42493f4/users/7389d11dbd964b87ae6ad774cbcfe16f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:17.177425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae378-b3e1-414a-b203-0df687b8765f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:17.178545 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae378-b3e1-414a-b203-0df687b8765f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:17.179208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ae378-b3e1-414a-b203-0df687b8765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:17.211224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c94ae378-b3e1-414a-b203-0df687b8765f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:17.212406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c94ae378-b3e1-414a-b203-0df687b8765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:17.279090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1255/2535] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:17.289246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:17.291460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:17.368883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a08ebf-2283-47ce-99e4-4847be9f5773 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:17.372830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:17.373568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:17.373979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:17.374382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:17.384571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:17.386312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:17.398012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1281/2536] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:18.118589 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.119096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.119182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.155576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] MFA Rules not processed for user `7389d11dbd964b87ae6ad774cbcfe16f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:18.158043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:18.277787 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1256/2537] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:18.282284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.300742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc443a-455b-4eb8-8211-6202bf83b22f service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.311923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.313220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.313559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.313933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.314639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.317410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.370198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1282/2538] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:18 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:18.370853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.387961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e286a9a-ea72-4ea4-bfed-f5cd07e5d3c7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.462905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.463654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.463984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.464322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.466143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.469410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.497542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1257/2539] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:18 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:05:18.501510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.512290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00329a-8f37-4914-b4c1-b88790d4e99d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.519426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.520043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.520320 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.520589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.521273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.523052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.535290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1283/2540] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:05:18.539346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.550417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73cf81-005d-4dd6-bcfa-a4628e390a53 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.555735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.556461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.556802 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.557148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] PATH_INFO: `/v3/projects/10e70075035a46d890e8148e6b999253/users/cb91368333ad412eae739a2b29fec43d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.574698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:18.579030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e70075035a46d890e8148e6b999253, user_id=cb91368333ad412eae739a2b29fec43d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.582151 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.599093 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1258/2541] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:18 2022] PUT /identity/v3/projects/10e70075035a46d890e8148e6b999253/users/cb91368333ad412eae739a2b29fec43d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:18.603331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.614680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6fa38-4508-4208-a25b-c7a491999ee7 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.680443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.680970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.681371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.681697 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.682489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.684240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49917cc-cfd4-4b03-8d76-9d445770bd21 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.698446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1284/2542] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:05:18.699862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:18.711763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefb352-4c00-4211-9a5e-be5590d9e35d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:18.717166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:18.717864 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:18.718156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:18.718434 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] PATH_INFO: `/v3/projects/10e70075035a46d890e8148e6b999253/users/cb91368333ad412eae739a2b29fec43d/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:18.720937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e70075035a46d890e8148e6b999253, user_id=cb91368333ad412eae739a2b29fec43d, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:18.723797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:18.735975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1259/2543] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:18 2022] PUT /identity/v3/projects/10e70075035a46d890e8148e6b999253/users/cb91368333ad412eae739a2b29fec43d/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:19.731638 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:19.732193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:19.732193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:19.771436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] MFA Rules not processed for user `cb91368333ad412eae739a2b29fec43d`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:19.772943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:19.815359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:19.894036 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1285/2544] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:19.911037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:19.913695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:19.986526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dcf26-c4a9-4163-8255-1868e4f09afc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:19.990915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:19.991538 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:19.991822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:19.992099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:20.060079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:20.061534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:20.072894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1260/2545] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:21.493629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:21.495101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:21.507867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973b2b8-2545-49cc-af75-5ea2efc6d3be None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:21.511717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:21.512345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:21.512623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:21.512890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:21.524084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:21.525652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973f01d-18d0-4a98-bd73-893a9134eb87 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:21.537015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1286/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:28.600838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:28.655911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b52c59-349e-4dde-a4e6-8d5f99a56fd9 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:28.659657 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:28.660263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:28.660536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:28.660804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:28.662567 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:28.672343 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:28.673772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81be1a-27ee-4431-98f9-6bf106f313df service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:28.682424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1261/2547] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:28.807705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:28.808234 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:28.808346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:28.809654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1287/2548] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:05:28.816375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e309-8ba9-4fb3-a9ef-43639f7e8541 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:28.816681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e309-8ba9-4fb3-a9ef-43639f7e8541 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:28.816962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75e309-8ba9-4fb3-a9ef-43639f7e8541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:28.836988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-fd75e309-8ba9-4fb3-a9ef-43639f7e8541 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:28.838224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-fd75e309-8ba9-4fb3-a9ef-43639f7e8541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:28.869394 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:28.919131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1262/2549] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:05:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:28.931051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:28.932856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:28.992449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ae71d-a4ab-48b8-bcea-cd7b4c6954f5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:28.995958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:28.996579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:28.996854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:28.997122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.007163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:29.008574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f004e1-b532-4d45-ac26-9b27fc9f99df service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:29.019068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1288/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:29.086503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90225-8f43-425e-804a-b5ddc0572463 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.086799 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90225-8f43-425e-804a-b5ddc0572463 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.087075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd90225-8f43-425e-804a-b5ddc0572463 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.088432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1263/2551] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:05:29.093602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3724-ecff-400d-914b-52b01888f07f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.093923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3724-ecff-400d-914b-52b01888f07f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.094204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f3724-ecff-400d-914b-52b01888f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.114469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-277f3724-ecff-400d-914b-52b01888f07f None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:29.115698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-277f3724-ecff-400d-914b-52b01888f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:29.122236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1289/2552] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:05:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:29.134662 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd90225-8f43-425e-804a-b5ddc0572463 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:29.145621 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd90225-8f43-425e-804a-b5ddc0572463 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:29.149331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:29.149957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.150231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.150499 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.158040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:29.159507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d688a-df72-4f74-a9fc-713af718ae94 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:29.167361 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1264/2553] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:29.252649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d8e7-7790-4daf-9403-5f31c7ae0eb2 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.252960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d8e7-7790-4daf-9403-5f31c7ae0eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.253258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d8e7-7790-4daf-9403-5f31c7ae0eb2 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.254652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1290/2554] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:05:29.259479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a747-6822-4527-acd1-18430c3f9daf None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.259762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a747-6822-4527-acd1-18430c3f9daf None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.260036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882a747-6822-4527-acd1-18430c3f9daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.280141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9882a747-6822-4527-acd1-18430c3f9daf None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:29.281412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9882a747-6822-4527-acd1-18430c3f9daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:29.290519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1265/2555] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:05:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:29.301316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6d8e7-7790-4daf-9403-5f31c7ae0eb2 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:29.312871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6d8e7-7790-4daf-9403-5f31c7ae0eb2 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:29.316470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:29.317048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:29.317322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:29.317590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:29.327939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:29.329443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:29.339986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1291/2556] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:31.647741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:31.648204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:31.648299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:31.666995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:31.668180 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:31.728020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1266/2557] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:05:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:31.740558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:31.742012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:31.810232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad83d9-79ab-4056-88c3-946b91a536b9 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:31.815764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:31.816374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:31.816646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:31.816913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:31.825358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:31.826952 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:31.834858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1292/2558] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:05:31.855833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:31.857223 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:31.873951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ea4c2-9d5b-447e-8992-228ed1cdf1e0 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:31.884457 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:31.885084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:31.885369 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:31.885625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:31.887583 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:31.969541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:31.971956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:31.999957 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1267/2559] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:34.571070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:34.572923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:34.610063 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:34.687729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93154a-2719-4189-afcf-5d10fa2a6ae4 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:34.693350 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:34.694000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:34.694275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:34.694543 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:34.696250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:34.722714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:34.730268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:34.757010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1293/2560] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:35.266861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:35.331558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:35.344515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef59a18-4553-47b6-9ef0-7f969629bc9b service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:35.350137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:35.351038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:35.351377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:35.351732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] PATH_INFO: `/v3/users/7389d11dbd964b87ae6ad774cbcfe16f` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:35.352945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7389d11dbd964b87ae6ad774cbcfe16f)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:35.354988 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:35.420175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7389d11dbd964b87ae6ad774cbcfe16f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.428702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7389d11dbd964b87ae6ad774cbcfe16f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.429528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7389d11dbd964b87ae6ad774cbcfe16f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.430231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7389d11dbd964b87ae6ad774cbcfe16f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.438632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7389d11dbd964b87ae6ad774cbcfe16f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.466382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1268/2561] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:35 2022] DELETE /identity/v3/users/7389d11dbd964b87ae6ad774cbcfe16f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:35.648883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:35.660568 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf161a2-d57d-4946-b0b1-ca59eff29ad9 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:35.742951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:35.743662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:35.743947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:35.744248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] PATH_INFO: `/v3/projects/0a6bd12fa7d04b8e866bdaffc42493f4` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:35.745460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6bd12fa7d04b8e866bdaffc42493f4)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:35.747479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:35.813091 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6bd12fa7d04b8e866bdaffc42493f4', 'request_id': 'req-2daf7f09-0177-42c8-8c8b-281f14f543d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.820067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6bd12fa7d04b8e866bdaffc42493f4', 'request_id': 'req-2daf7f09-0177-42c8-8c8b-281f14f543d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.829104 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6bd12fa7d04b8e866bdaffc42493f4', 'request_id': 'req-2daf7f09-0177-42c8-8c8b-281f14f543d8'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.840633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] The token cache is being invalidate because project 0a6bd12fa7d04b8e866bdaffc42493f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:35.840943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.841566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:35.843245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1294/2562] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:35 2022] DELETE /identity/v3/projects/0a6bd12fa7d04b8e866bdaffc42493f4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:35.847480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:35.848932 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:35.862910 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c53d21-2240-4047-8efd-84cd48d2bfb8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:35.932477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:35.933317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:35.933602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:35.933899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] PATH_INFO: `/v3/users/cb91368333ad412eae739a2b29fec43d` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:35.935131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb91368333ad412eae739a2b29fec43d)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:35.937890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:35.991840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb91368333ad412eae739a2b29fec43d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.002120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb91368333ad412eae739a2b29fec43d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.003141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb91368333ad412eae739a2b29fec43d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.003978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb91368333ad412eae739a2b29fec43d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.012609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-5d733127-f10d-48b4-9473-d7f4a9e7c8a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb91368333ad412eae739a2b29fec43d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.020688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:36.033935 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1269/2563] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:35 2022] DELETE /identity/v3/users/cb91368333ad412eae739a2b29fec43d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:36.296489 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.297986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:36.338953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf7f09-0177-42c8-8c8b-281f14f543d8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.391536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:36.406790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.407424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.407731 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.408003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] PATH_INFO: `/v3/projects/10e70075035a46d890e8148e6b999253` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.409145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e70075035a46d890e8148e6b999253)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.411098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.465647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e70075035a46d890e8148e6b999253', 'request_id': 'req-3b7223b5-7db3-49a4-97e1-f8eb58b60687'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.473567 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e70075035a46d890e8148e6b999253', 'request_id': 'req-3b7223b5-7db3-49a4-97e1-f8eb58b60687'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.487119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e70075035a46d890e8148e6b999253', 'request_id': 'req-3b7223b5-7db3-49a4-97e1-f8eb58b60687'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.510892 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] The token cache is being invalidate because project 10e70075035a46d890e8148e6b999253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:36.511185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.511846 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3b7223b5-7db3-49a4-97e1-f8eb58b60687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:36.516933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1295/2564] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:36 2022] DELETE /identity/v3/projects/10e70075035a46d890e8148e6b999253 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:36.518769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.519234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.519573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.540297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:36.541501 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:36.607472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1270/2565] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:36.615050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.615362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.615609 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.637432 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:36.638654 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:36.658411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1296/2566] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:36.659338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.670773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8629fe5-dac5-4504-9f6d-07a3573124ef None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.675971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.676580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.676858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.677156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.677920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:05:36.678230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.680195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.693491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1271/2567] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:05:36.698060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.707837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3e0ee-d2ea-4035-ad04-768c0884b426 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.712526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.713116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.713395 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.713665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.714332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.716283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.752152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1297/2568] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:36.755976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.769164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c0d33-061d-45b7-ad3c-11b2208a4c68 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.833678 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.834876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.835393 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.835766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.836723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.838817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.867293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1272/2569] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:36 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:36.870912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.886154 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be53a6-c5df-48b2-9bab-8ad64b363507 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.893762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.894868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.895478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.896074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.896916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.898692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.911351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1298/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:36.915147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.936204 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6a7bf-34ee-4cfd-a725-9beff2b36bc3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.944966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.946070 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.946417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.946949 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:36.948132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:36.950498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53fd5a-ee6c-4994-8832-ead072734f90 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:36.964631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1273/2571] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:36.968829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:36.979624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73878e7a-1a31-46e8-ae40-9951508c67f5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:36.984717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:36.985358 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:36.985633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:36.985942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] PATH_INFO: `/v3/projects/cc691922b0334874b0a2da0b95648c46/users/5aeee93f6435485093de4c309c986206/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:37.001587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc691922b0334874b0a2da0b95648c46, user_id=5aeee93f6435485093de4c309c986206, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:37.005321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:37.021970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1299/2572] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:36 2022] PUT /identity/v3/projects/cc691922b0334874b0a2da0b95648c46/users/5aeee93f6435485093de4c309c986206/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:37.033838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959e4-acf1-425b-9162-80d913c1984a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:37.034243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959e4-acf1-425b-9162-80d913c1984a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:37.034623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815959e4-acf1-425b-9162-80d913c1984a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:37.075596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-815959e4-acf1-425b-9162-80d913c1984a None None] MFA Rules not processed for user `5aeee93f6435485093de4c309c986206`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:37.076828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-815959e4-acf1-425b-9162-80d913c1984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:37.125337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:37.148421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1274/2573] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:37.159736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:37.161430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:37.232840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308ef13-c752-4291-b49f-a0c4bfa006ae admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:37.237214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:37.237882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:37.238162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:37.238430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:37.246955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:37.248373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:37.258206 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1300/2574] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:05:47.145400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:47.253081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:54.813060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:54.813550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:54.813640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:54.837953 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:54.840751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:54.906992 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1275/2575] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:05:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:54.921293 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:54.922832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:54.995413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5884-3b31-4d05-8dea-702d89081494 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:55.000968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:55.001576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:55.001903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:55.002172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:55.010191 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:55.011768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c4bd6-0b58-461e-a1b8-5c864d5ba49c service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:55.019837 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1301/2576] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:05:57.121002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:57.185504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58460f-b54d-4be2-9519-24eecb29ff4a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:57.190466 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:57.191199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:57.191368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:57.191643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] PATH_INFO: `/v3/users/5aeee93f6435485093de4c309c986206` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:57.192808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aeee93f6435485093de4c309c986206)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:57.194971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:57.248958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aeee93f6435485093de4c309c986206'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:57.258872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aeee93f6435485093de4c309c986206'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:57.259633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aeee93f6435485093de4c309c986206'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:57.260237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aeee93f6435485093de4c309c986206'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:57.268331 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aeee93f6435485093de4c309c986206'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:57.294616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1276/2577] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:05:57 2022] DELETE /identity/v3/users/5aeee93f6435485093de4c309c986206 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:57.300987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6588-7e32-4334-8814-c4255709a746 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:57.301936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6588-7e32-4334-8814-c4255709a746 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:57.302305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d6588-7e32-4334-8814-c4255709a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:57.321995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-860d6588-7e32-4334-8814-c4255709a746 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:57.323463 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-860d6588-7e32-4334-8814-c4255709a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:57.383734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1302/2578] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:05:57.394148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:57.395562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:57.423712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:57.458520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10d1d-9521-43e0-a43f-629d9e5c5a85 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:57.462258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:57.462874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:57.463158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:57.463428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:57.472968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:57.474436 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d7618-1ad0-4b54-8fb3-c1948f5211ce service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:57.484562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1277/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:05:57.921773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d6588-7e32-4334-8814-c4255709a746 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:57.923580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-860d6588-7e32-4334-8814-c4255709a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:57.939124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d6588-7e32-4334-8814-c4255709a746 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:57.945475 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:57.946812 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:57.947467 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:57.948172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] PATH_INFO: `/v3/projects/cc691922b0334874b0a2da0b95648c46` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:57.950287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc691922b0334874b0a2da0b95648c46)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:57.953404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.001477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc691922b0334874b0a2da0b95648c46', 'request_id': 'req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:58.008379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc691922b0334874b0a2da0b95648c46', 'request_id': 'req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:58.017000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc691922b0334874b0a2da0b95648c46', 'request_id': 'req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:58.028988 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] The token cache is being invalidate because project cc691922b0334874b0a2da0b95648c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:05:58.029277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:58.030533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-3dfc9c98-9f2f-49b7-9022-1b80758e6edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:05:58.032018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1303/2580] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:05:57 2022] DELETE /identity/v3/projects/cc691922b0334874b0a2da0b95648c46 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:05:58.036886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.037172 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.037445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.056801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:58.058035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:58.124978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1278/2581] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:58.132469 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.132782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.133056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.157140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:58.158396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:58.171509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1304/2582] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:58.172698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.183108 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed88ee-7085-4d00-af5c-2432780ca34e None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.188810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.189576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.190022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.190417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.191035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:05:58.191404 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.193081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.212213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1279/2583] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:05:58.213391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.223632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97912bb-1b13-4b14-8464-9abc5c20faea None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.228359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.228960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.229236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.229504 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.230214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.232099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.237418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:58.262710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1305/2584] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:05:58 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:05:58.266941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.276976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18d90-e48a-4880-9be6-57251c083e76 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.350382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.351031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.351318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.351840 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.352500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.354732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.383163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1280/2585] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:05:58 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:05:58.384290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.395680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93372d-221f-4112-b9a8-768685bf7653 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.401428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.402193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.402583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.403310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.404069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.406322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.417948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1306/2586] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:58.421574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.432017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d2be0-8ef2-40c7-87d5-527142959bb3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.437119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.437716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.438021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.438306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.438931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.440530 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc272d-89e0-45c3-9fab-cb5161d63f88 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.457794 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1281/2587] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:05:58.459570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.471147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add910e-0a74-48a7-a793-3186e44b3bf8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.476249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.477001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.477353 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.477701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] PATH_INFO: `/v3/projects/ed4b184a9a25464ebba65e608fd25749/users/4823cec6db47401f8637fac4fbd62a3f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.496502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4b184a9a25464ebba65e608fd25749, user_id=4823cec6db47401f8637fac4fbd62a3f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.500144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.516416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1307/2588] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:05:58 2022] PUT /identity/v3/projects/ed4b184a9a25464ebba65e608fd25749/users/4823cec6db47401f8637fac4fbd62a3f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:05:58.527173 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.527461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.527790 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.577408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] MFA Rules not processed for user `4823cec6db47401f8637fac4fbd62a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:05:58.578711 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:58.604916 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:05:58.658729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1282/2589] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:05:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:05:58.670929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:05:58.673067 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:05:58.750156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a241d-4e35-4d27-8e78-c04cd6f213e3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:05:58.754429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:05:58.755102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:05:58.755545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:05:58.755847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:05:58.766730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:05:58.768144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:05:58.777518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1308/2590] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:05:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:02.747482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:02.749372 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:02.804689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3720da3-60a8-4348-94a3-ede452386cc2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:02.808357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:02.808959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:02.809239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:02.809507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:02.811767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:02.873647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:02.875121 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:02.885360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1283/2591] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:06.065707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.067588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:06.087002 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe8b6-9deb-4629-aaeb-5cdc9547d3fc service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.096585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.097296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.097604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.097930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] PATH_INFO: `/v3/users/6a3c9d9bf3c8455aa93ad43b10bfd956` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.099101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3c9d9bf3c8455aa93ad43b10bfd956)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.101068 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.147438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3c9d9bf3c8455aa93ad43b10bfd956'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.155112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3c9d9bf3c8455aa93ad43b10bfd956'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.155734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3c9d9bf3c8455aa93ad43b10bfd956'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.156323 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3c9d9bf3c8455aa93ad43b10bfd956'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.163364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a9ce18bd-a5ec-41c3-98bc-6fdd1e650279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3c9d9bf3c8455aa93ad43b10bfd956'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.180328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1309/2592] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:06 2022] DELETE /identity/v3/users/6a3c9d9bf3c8455aa93ad43b10bfd956 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:06.347351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.359237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd63662-129a-4f18-9b55-5d8f7e40bebe service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.427306 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.427934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.428214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.428564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] PATH_INFO: `/v3/projects/189468d0e3034caaa498442603da0609` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.429689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] RBAC: Authorizing `identity:delete_project(project_id=189468d0e3034caaa498442603da0609)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.431612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.448028 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=100 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:06.478169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189468d0e3034caaa498442603da0609', 'request_id': 'req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.484234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189468d0e3034caaa498442603da0609', 'request_id': 'req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.491406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189468d0e3034caaa498442603da0609', 'request_id': 'req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.502604 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] The token cache is being invalidate because project 189468d0e3034caaa498442603da0609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:06.502889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.503517 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f7a369c3-13e7-4508-bd8c-af7f2d17f7df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:06.504984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1284/2593] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:06 2022] DELETE /identity/v3/projects/189468d0e3034caaa498442603da0609 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:06.510033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.510345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.510644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.531751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:06.532977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:06.551716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:06.596630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1310/2594] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:06.604409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.604739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.605017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.625515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:06.626812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:06.635191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1285/2595] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:06.639488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.648986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fcdf0-69ad-4336-a73d-cf44d92a2b76 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.653492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.654634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.654933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.655202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.655844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:06:06.656140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.657719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.669405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1311/2596] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:06:06.675430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.685764 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4913a5-5216-4081-b1fe-6cf70c0d157c None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.690745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.691390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.691671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.691943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.692588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.694548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.726649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1286/2597] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:06 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:06.730720 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.741354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe2245-29f6-45bc-8aca-f18acdaf4a97 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.797721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.798383 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.798665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.798940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.799573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.801518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.824799 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1312/2598] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:06 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:06.829333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.849161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17225073-8e55-4f88-b9c5-33144d477d70 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.849161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.849161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.849161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.849546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.849546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.849546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.858416 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1287/2599] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:06:06.862311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.872149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fe4b4-89f6-42d4-9294-953afe400587 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.876758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.877356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.877614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.877920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.878552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.880138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b627ac-d755-40ee-a2a3-98a0102b3f1b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.891834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1313/2600] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:06.896215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:06.906939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f1b8d-8311-4799-b57b-eee6445d5208 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:06.911920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:06.912578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.912861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.913139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] PATH_INFO: `/v3/projects/c5eff8172c924e9b94fc74de4212afa4/users/92f3a51ce0704b7ebb4760bf63ac391a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.928049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5eff8172c924e9b94fc74de4212afa4, user_id=92f3a51ce0704b7ebb4760bf63ac391a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:06.930760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:06.944155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1288/2601] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:06 2022] PUT /identity/v3/projects/c5eff8172c924e9b94fc74de4212afa4/users/92f3a51ce0704b7ebb4760bf63ac391a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:06.956075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:06.956373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:06.956637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:06.984878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] MFA Rules not processed for user `92f3a51ce0704b7ebb4760bf63ac391a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:06.986226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:07.059032 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1314/2602] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:07.063947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.125665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f91c1-b890-4c9d-b917-efeda88cdc2a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.130231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.130828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.131103 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.131370 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.132007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.133919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.160976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1289/2603] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:07 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:07.165084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.175514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5fda2-c2f8-48b3-8db1-f9549d9a1a8e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.191936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:07.232263 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.233455 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.233745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.234028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.234660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.236574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.257312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1315/2604] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:07 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:06:07.261425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.271505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10c31-e481-4f92-97cd-a017c8e3b8dd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.276066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.276665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.276955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.277209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.277876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.279954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.290195 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1290/2605] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:07.294178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.304580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e10bc-8dd3-4575-a11b-416c689bf9c8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.309137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.309804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.310085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.310374 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] PATH_INFO: `/v3/projects/75135d79c74f4e17b9e08963cb21507b/users/cb41bcc87af14a2bab9f78b405b61e83/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.325019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75135d79c74f4e17b9e08963cb21507b, user_id=cb41bcc87af14a2bab9f78b405b61e83, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.327696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.340402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1316/2606] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:07 2022] PUT /identity/v3/projects/75135d79c74f4e17b9e08963cb21507b/users/cb41bcc87af14a2bab9f78b405b61e83/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:07.344983 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.355676 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fa012-0cef-48b8-8c2d-c0dbf1641f37 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.366822 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:07.415086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.415611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.415891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.416164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.416835 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.418511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6447e-03e7-4452-a479-5dce83a1132b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.428731 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1291/2607] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:07.432953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.443250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116f782-1bc7-443b-a76c-8e9abe869beb admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.448287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.448945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.449222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.449494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] PATH_INFO: `/v3/projects/75135d79c74f4e17b9e08963cb21507b/users/cb41bcc87af14a2bab9f78b405b61e83/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.451431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75135d79c74f4e17b9e08963cb21507b, user_id=cb41bcc87af14a2bab9f78b405b61e83, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.454089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.464080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1317/2608] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:07 2022] PUT /identity/v3/projects/75135d79c74f4e17b9e08963cb21507b/users/cb41bcc87af14a2bab9f78b405b61e83/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:07.475865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.476152 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.476458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.503943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] MFA Rules not processed for user `cb41bcc87af14a2bab9f78b405b61e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:07.505140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:07.570739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1292/2609] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:07.582918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.584602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:07.640987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfbbb0-f321-4bee-91ea-cdf9b7dc0cc9 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.644583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.645177 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.645449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.645716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.655109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.656521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60e47a-a9c3-470e-9905-325ea7393111 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.665788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1318/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:06:07.739858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:07.749480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d3f5b-aad7-43f2-a3d2-c729d6db05f2 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:07.753120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:07.753716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:07.754033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:07.754305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:07.810149 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:07.811575 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98acb4e-6803-4f80-b7f2-0ec2896d779c service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:07.820196 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1293/2611] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:17.661722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:17.816305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=85 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:28.316676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.317253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.317253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.318938 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1319/2612] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:06:28.325528 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6727-3d96-41e0-b0d3-b57b593a6993 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.326022 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6727-3d96-41e0-b0d3-b57b593a6993 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.326243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b6727-3d96-41e0-b0d3-b57b593a6993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.358792 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-be2b6727-3d96-41e0-b0d3-b57b593a6993 None None] MFA Rules not processed for user `9e0abf2c4f1a45b88cf4f54237eb0def`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:28.361075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-be2b6727-3d96-41e0-b0d3-b57b593a6993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:28.425283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1294/2613] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:06:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:28.445632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:28.448281 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:28.467690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1f776-0b58-43e3-a63b-3777762a99d1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:28.471640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:28.472274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.472603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.472986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.480634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:28.482771 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4bceb-5db7-4641-9086-5941404b0ff6 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:28.491754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1320/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:28.571620 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16a351-e02d-42e4-ae48-b1fe5a6635d8 None None] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.571909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16a351-e02d-42e4-ae48-b1fe5a6635d8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.572467 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16a351-e02d-42e4-ae48-b1fe5a6635d8 None None] PATH_INFO: `/` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.573853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1295/2615] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:06:28.579629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.579908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.580175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.600088 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:28.601274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:28.667364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1321/2616] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:06:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:28.678439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16a351-e02d-42e4-ae48-b1fe5a6635d8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:28.689901 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16a351-e02d-42e4-ae48-b1fe5a6635d8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:28.693547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:28.694178 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:28.694468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:28.694742 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:28.705410 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:28.706899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e1bdd-3991-4000-b85d-0af0476860ab service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:28.718334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1296/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:06:38.657957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=37 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:38.712923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=29 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:40.653888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:40.655356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:40.723254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef61ed-193a-4601-94b3-75c49b631d16 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:40.727903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:40.728917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:40.728917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:40.729259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] PATH_INFO: `/v3/users/4823cec6db47401f8637fac4fbd62a3f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:40.730505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4823cec6db47401f8637fac4fbd62a3f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:40.732993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:40.780825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4823cec6db47401f8637fac4fbd62a3f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:40.788209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4823cec6db47401f8637fac4fbd62a3f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:40.788931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4823cec6db47401f8637fac4fbd62a3f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:40.789550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4823cec6db47401f8637fac4fbd62a3f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:40.797552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4823cec6db47401f8637fac4fbd62a3f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:40.816361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1322/2618] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:40 2022] DELETE /identity/v3/users/4823cec6db47401f8637fac4fbd62a3f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:40.820836 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:40.821337 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:40.821459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:40.973132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:40.974448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:41.040382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1297/2619] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:41.052785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:41.055099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:41.121119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a59e-2ca8-4e6c-b6b6-7d1475801b42 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:41.125404 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:41.126056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.126335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.126610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.136621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:41.138311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc096f3-7dba-4714-b5e1-014e453c3c29 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:41.148708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1323/2620] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:41.570656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:41.572366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:41.588971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02fb6-720b-41fd-9a70-29ceb5828079 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:41.597044 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:41.597952 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.598228 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.598544 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] PATH_INFO: `/v3/projects/ed4b184a9a25464ebba65e608fd25749` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.599959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4b184a9a25464ebba65e608fd25749)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:41.605599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:41.656602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4b184a9a25464ebba65e608fd25749', 'request_id': 'req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:41.663001 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4b184a9a25464ebba65e608fd25749', 'request_id': 'req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:41.671862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4b184a9a25464ebba65e608fd25749', 'request_id': 'req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:41.696043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] The token cache is being invalidate because project ed4b184a9a25464ebba65e608fd25749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:41.696347 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:41.697321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-92056fd7-203e-47c5-8e9d-46e3c1d64bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:41.698928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1298/2621] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:41 2022] DELETE /identity/v3/projects/ed4b184a9a25464ebba65e608fd25749 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:41.703858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.704328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.704449 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.724270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:41.725497 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:41.742946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:41.787322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1324/2622] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:06:41.795745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.796021 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.796290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.816145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:41.817371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:41.825643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1299/2623] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:06:41.830057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:41.839899 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc5f09-f4e6-4ace-b793-81415f22df03 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:41.845161 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:41.845917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.846193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.846465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.847110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:06:41.847407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:41.849016 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:41.866184 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1325/2624] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:06:41.867688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:41.877050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d838dd0-b8f7-4c4d-9ea2-e66a591a40a9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:41.881710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:41.882770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.883048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.883319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.883954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:41.885912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:41.913902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1300/2625] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:41.918147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:41.928874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66a029-72d5-4ab9-93b4-4d26558ebadc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:41.994481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:41.995283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:41.995588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:41.995902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:41.996676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:41.998905 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.022450 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1326/2626] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:41 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:42.023827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.034650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77126cf5-ec46-40db-a563-d6de525be5e7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.039422 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.040017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.040294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.040565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.041203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.042821 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.057430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1301/2627] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:42.058852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.070134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1406d02-195a-40d2-8eb9-e0742f3c8809 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.075583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.076203 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.076540 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.076811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.077589 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.079199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bba00-9889-4083-a241-348fee2fa896 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.090020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1327/2628] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:06:42.094905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.104691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda8f0f-58d3-41dd-8f2f-ccc56f9494fd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.110144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.110805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.111084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.111356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] PATH_INFO: `/v3/projects/b6d2f5b03a5340bb936885199c296aa4/users/170141aed8b641f3bcf4e7fdf66327c3/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.127084 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d2f5b03a5340bb936885199c296aa4, user_id=170141aed8b641f3bcf4e7fdf66327c3, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.129761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.144485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1302/2629] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:42 2022] PUT /identity/v3/projects/b6d2f5b03a5340bb936885199c296aa4/users/170141aed8b641f3bcf4e7fdf66327c3/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:42.156204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.156490 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.156762 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.185148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] MFA Rules not processed for user `170141aed8b641f3bcf4e7fdf66327c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:42.186379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:42.254367 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1328/2630] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:42.265797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.267582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:42.289419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:42.331656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c078-ac60-4cd9-9926-b088c9bec7ec admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.336142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.336843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.337188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.337529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.347451 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.349239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55ca7c-86dd-414e-a42e-25825ec7be37 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.359128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1303/2631] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:42.385001 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.405118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:42.438485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a43a7-4fc3-4caf-b3f5-db43f67740f2 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.438767 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a43a7-4fc3-4caf-b3f5-db43f67740f2 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.439038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a43a7-4fc3-4caf-b3f5-db43f67740f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.456341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b7206-0f14-4bfd-be4d-e631e99605dc None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.458468 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-917a43a7-4fc3-4caf-b3f5-db43f67740f2 None None] MFA Rules not processed for user `a4828b30dda248758b4161eafe8668c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:42.459760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-917a43a7-4fc3-4caf-b3f5-db43f67740f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:42.461120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.461786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.462070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.462348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] PATH_INFO: `/v3/users/170141aed8b641f3bcf4e7fdf66327c3` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.463495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170141aed8b641f3bcf4e7fdf66327c3)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.465426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.515244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170141aed8b641f3bcf4e7fdf66327c3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:42.520842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1304/2632] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:06:42.523008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170141aed8b641f3bcf4e7fdf66327c3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:42.523707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170141aed8b641f3bcf4e7fdf66327c3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:42.524311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170141aed8b641f3bcf4e7fdf66327c3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:42.532183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170141aed8b641f3bcf4e7fdf66327c3'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:42.550700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1329/2633] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:42 2022] DELETE /identity/v3/users/170141aed8b641f3bcf4e7fdf66327c3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:42.555529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.555529 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.555716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.581599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:42.582890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:42.646520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1305/2634] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:42.657108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:42.658619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:42.723677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eead09-07a9-49be-9c4d-ffe675e65bb1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:42.728230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:42.729020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:42.729493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:42.729867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:42.740102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:42.741541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f02c02-3c9e-4a48-8de9-60c7ad5c07f7 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:42.751588 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1330/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:43.119950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.121611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:43.133522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460807f-6406-4683-87cb-26a1390e31b8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.139286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.140017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.146903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.147342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] PATH_INFO: `/v3/projects/b6d2f5b03a5340bb936885199c296aa4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.149079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d2f5b03a5340bb936885199c296aa4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.151257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.219779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d2f5b03a5340bb936885199c296aa4', 'request_id': 'req-91aa5bd6-1750-4067-9219-e4909d6164d6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:43.226335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d2f5b03a5340bb936885199c296aa4', 'request_id': 'req-91aa5bd6-1750-4067-9219-e4909d6164d6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:43.235701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d2f5b03a5340bb936885199c296aa4', 'request_id': 'req-91aa5bd6-1750-4067-9219-e4909d6164d6'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:43.243156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:43.256092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] The token cache is being invalidate because project b6d2f5b03a5340bb936885199c296aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:43.256761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:43.257514 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-91aa5bd6-1750-4067-9219-e4909d6164d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:43.259938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1306/2636] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:43 2022] DELETE /identity/v3/projects/b6d2f5b03a5340bb936885199c296aa4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:06:43.265721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.266232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.266586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.288305 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:43.289647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:43.330318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:43.490169 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1331/2637] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:43.557351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.557893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.558263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.582279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:43.583695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:43.591980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1307/2638] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:43.596580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.609315 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ad96e-8810-49fe-a937-15500c4db1ee None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.612699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.613294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.613566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.613863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.614502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:06:43.614795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.616360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.628984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1332/2639] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:06:43.633352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.642504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a4f12-ab46-4e4f-9ba5-69ffd717c3ea None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.647170 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.647763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.648036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.648304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.648934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.651307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.678613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1308/2640] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:43 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:43.683064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.694479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5e724-f63b-4daa-98cd-2a9d20411b65 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.752768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.753792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.754178 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.754539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.755345 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.757338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.778192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1333/2641] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:43 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:43.782309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.792631 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c38bd7-fb3f-4002-a173-d78fcc80f7de admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.798542 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.799081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.799373 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.799625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.800262 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.801898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.812770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1309/2642] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:43.816749 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.827147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ede4d-9e1c-49ab-be75-33b8ce7fc766 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.831648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.832236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.832512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.832777 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.833412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.835029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147f5c-548e-4a36-9fe6-1bd80f75c41f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.845351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1334/2643] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:43.849214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:43.859450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16948617-7388-49ad-9a0a-3b4451eed44c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:43.864029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:43.864661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.864939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.865209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] PATH_INFO: `/v3/projects/4bd7c7df649140b296a9febd2ac937d7/users/94ae3b9f39cd481c88c617e1e1463201/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.879911 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd7c7df649140b296a9febd2ac937d7, user_id=94ae3b9f39cd481c88c617e1e1463201, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:43.882581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:43.895797 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1310/2644] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:43 2022] PUT /identity/v3/projects/4bd7c7df649140b296a9febd2ac937d7/users/94ae3b9f39cd481c88c617e1e1463201/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:43.906465 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:43.906745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:43.907012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:43.933951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] MFA Rules not processed for user `94ae3b9f39cd481c88c617e1e1463201`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:43.935715 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:44.000620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1335/2645] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:44.011344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:44.013663 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:44.069414 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f332c-a352-49b1-920d-4be3fbc253d2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:44.073267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:44.073909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:44.074185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:44.074452 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:44.083194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:44.084614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70564be2-1062-45f5-97b3-5bee0ca0a44b service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:44.094192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1311/2646] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:49.865533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:49.867030 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:49.906928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:49.933541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec43b16-dfb6-40c0-9421-12a41a08bb3e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:49.937992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:49.938610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:49.938884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:49.939494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] PATH_INFO: `/v3/users/92f3a51ce0704b7ebb4760bf63ac391a` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:49.940623 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f3a51ce0704b7ebb4760bf63ac391a)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:49.942542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:49.986830 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f3a51ce0704b7ebb4760bf63ac391a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:49.993700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f3a51ce0704b7ebb4760bf63ac391a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:49.994352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f3a51ce0704b7ebb4760bf63ac391a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:49.994948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f3a51ce0704b7ebb4760bf63ac391a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.001691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f3a51ce0704b7ebb4760bf63ac391a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.018492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1336/2647] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:49 2022] DELETE /identity/v3/users/92f3a51ce0704b7ebb4760bf63ac391a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:50.023157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:50.023636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:50.023755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:50.042897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:50.044716 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:50.101922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1312/2648] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:50.112121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:50.113538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:50.169948 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ada35-3e49-4917-b681-eb4092b174d8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:50.173580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:50.174231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:50.174506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:50.174778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:50.186716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:50.188173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:50.198674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1337/2649] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:50.515640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:50.517192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:50.528981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842db1e-b7d8-49e0-b8a4-3606b6da40c5 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:50.533814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:50.534677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:50.536140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:50.536437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] PATH_INFO: `/v3/projects/c5eff8172c924e9b94fc74de4212afa4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:50.537704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5eff8172c924e9b94fc74de4212afa4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:50.539642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:50.572316 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:50.589688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5eff8172c924e9b94fc74de4212afa4', 'request_id': 'req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.595897 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5eff8172c924e9b94fc74de4212afa4', 'request_id': 'req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.602984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5eff8172c924e9b94fc74de4212afa4', 'request_id': 'req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.613857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] The token cache is being invalidate because project c5eff8172c924e9b94fc74de4212afa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:50.614148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.614789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.616326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1313/2650] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:50 2022] DELETE /identity/v3/projects/c5eff8172c924e9b94fc74de4212afa4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:50.620269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:50.629718 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64013436-038d-4c98-ac2a-846102e6de39 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:50.679928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:50.687461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:50.688726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:50.689015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:50.689582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] PATH_INFO: `/v3/users/cb41bcc87af14a2bab9f78b405b61e83` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:50.691054 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb41bcc87af14a2bab9f78b405b61e83)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:50.693148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:50.737723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb41bcc87af14a2bab9f78b405b61e83'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.744936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb41bcc87af14a2bab9f78b405b61e83'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.745546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb41bcc87af14a2bab9f78b405b61e83'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.746179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb41bcc87af14a2bab9f78b405b61e83'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.753598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-95eaa224-d115-490b-897b-c1f3fd5f323e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb41bcc87af14a2bab9f78b405b61e83'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:50.773043 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1338/2651] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:50 2022] DELETE /identity/v3/users/cb41bcc87af14a2bab9f78b405b61e83 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:51.031793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.033568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:51.056827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da9c6f-e3ac-49a5-9f96-59b6e8cc8f6f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.116636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621598b-e362-438f-9399-188fd408264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.117269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621598b-e362-438f-9399-188fd408264c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.117547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621598b-e362-438f-9399-188fd408264c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.117845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621598b-e362-438f-9399-188fd408264c admin admin] PATH_INFO: `/v3/projects/75135d79c74f4e17b9e08963cb21507b` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.118973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621598b-e362-438f-9399-188fd408264c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75135d79c74f4e17b9e08963cb21507b)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.120854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621598b-e362-438f-9399-188fd408264c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.166080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75135d79c74f4e17b9e08963cb21507b', 'request_id': 'req-1621598b-e362-438f-9399-188fd408264c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.172494 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75135d79c74f4e17b9e08963cb21507b', 'request_id': 'req-1621598b-e362-438f-9399-188fd408264c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.179667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75135d79c74f4e17b9e08963cb21507b', 'request_id': 'req-1621598b-e362-438f-9399-188fd408264c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.189962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] The token cache is being invalidate because project 75135d79c74f4e17b9e08963cb21507b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:51.190270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.190878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1621598b-e362-438f-9399-188fd408264c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.192411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1314/2652] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:51 2022] DELETE /identity/v3/projects/75135d79c74f4e17b9e08963cb21507b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:51.197510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.198015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.198113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.217902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:51.219083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:51.276673 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1339/2653] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:51.285726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.286030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.286309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.308412 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:51.310131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:51.318089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1315/2654] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:51.322530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.332108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4661d4-ee89-4bd3-9e9e-93c46b20bbdb None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.336577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.337162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.337464 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.337757 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.338394 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:06:51.338690 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.340258 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.351365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1340/2655] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:06:51.355403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.365330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db2c0-e25d-4b1a-9089-540f6b7a607a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.369989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.370576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.370852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.371118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.372381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.374295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.401547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1316/2656] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:51.406409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.416420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19d218-099f-4a59-bde2-d0364d57c68c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.474788 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.475527 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.475861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.476073 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.476706 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.478645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.490304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:51.507980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1341/2657] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:51 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:06:51.514038 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.525099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307ad60-49af-484a-987e-9e346eb38e92 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.532648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.533505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.533776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.534115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.534830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.536588 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.557842 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1317/2658] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:51.562627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.573529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d130-ea57-4409-bce7-ec3e49b75624 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.578354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.578951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.579227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.579496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.580141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.581755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77f70b-6d59-492e-a8e1-e8012ae130a2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.593390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1342/2659] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:06:51.597283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.603861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:51.608140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6d4f6-e5d9-4a4f-893d-77b61ca6c698 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.614746 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.615660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.616282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.616547 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] PATH_INFO: `/v3/projects/8c30742c604f4bf0aae5398962cdb8f1/users/1ea1cb7584ab4c228eff5cd2705629bc/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.632003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c30742c604f4bf0aae5398962cdb8f1, user_id=1ea1cb7584ab4c228eff5cd2705629bc, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.634775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.648781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1318/2660] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:51 2022] PUT /identity/v3/projects/8c30742c604f4bf0aae5398962cdb8f1/users/1ea1cb7584ab4c228eff5cd2705629bc/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:06:51.659564 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.659844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.660109 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.687881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] MFA Rules not processed for user `1ea1cb7584ab4c228eff5cd2705629bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:51.689204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:51.753048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1343/2661] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:51.764133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.766504 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:51.826695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193cb32-d5c8-4167-a6bf-221f0075f393 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.830346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.830950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.831231 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.831500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.842259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.843683 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e5da9-ad11-4996-8b30-bf10204ca5a9 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.853375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1319/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:06:51.874596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:51.935957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980392b-3fd9-4cd3-908e-aaa8874b2321 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:51.940514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:51.941131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:51.941402 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:51.941665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] PATH_INFO: `/v3/users/1ea1cb7584ab4c228eff5cd2705629bc` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:51.942845 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea1cb7584ab4c228eff5cd2705629bc)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:51.944738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:51.990515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea1cb7584ab4c228eff5cd2705629bc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.997410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea1cb7584ab4c228eff5cd2705629bc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.998058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea1cb7584ab4c228eff5cd2705629bc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:51.998672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea1cb7584ab4c228eff5cd2705629bc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.005413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea1cb7584ab4c228eff5cd2705629bc'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.013994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:52.025162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1344/2663] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:06:51 2022] DELETE /identity/v3/users/1ea1cb7584ab4c228eff5cd2705629bc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:52.030492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.030775 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.031041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.049758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:52.051238 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:52.119407 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1320/2664] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:52.126738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.128202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:52.199466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c149-5cdf-4867-9efb-c79ea18a3a45 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.203946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.204636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.205085 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.205267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.215680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.217123 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fda0d-3857-4e9d-ab12-e36309d93efb service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.233974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1345/2665] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:06:52.514105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.515574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:52.527296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906e32d-7750-4aa4-aa1f-e415c3c36285 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.531945 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.532645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.532984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.533342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] PATH_INFO: `/v3/projects/8c30742c604f4bf0aae5398962cdb8f1` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.534804 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c30742c604f4bf0aae5398962cdb8f1)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.536856 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.585440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c30742c604f4bf0aae5398962cdb8f1', 'request_id': 'req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.591908 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c30742c604f4bf0aae5398962cdb8f1', 'request_id': 'req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.600350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c30742c604f4bf0aae5398962cdb8f1', 'request_id': 'req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.612811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] The token cache is being invalidate because project 8c30742c604f4bf0aae5398962cdb8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:06:52.613106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.613719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-b9e643e0-cc74-4f31-b8e6-54b6e12d319e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:06:52.615482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1321/2666] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:06:52 2022] DELETE /identity/v3/projects/8c30742c604f4bf0aae5398962cdb8f1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:06:52.620183 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.620535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.620811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.640973 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:52.642309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:52.725831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1346/2667] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:52.732647 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.732927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.733199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.751119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:52.766800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:52.768150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:52.776554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1322/2668] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:52.780817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.790637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b2f9-8fde-4a55-ba57-729e796b2ea0 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.795933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.796535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.796816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.797086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.797756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:06:52.798055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.799658 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.812004 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1347/2669] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:06:52.816286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.826158 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d23d9-25ec-45b9-b3cb-a2b41845dc50 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.830993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.831692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.831972 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.832235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.832862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.834784 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.862540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1323/2670] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:06:52 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:52.866621 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.876389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386f53f-0df6-40a1-96d9-5cd8003e4722 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.932895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.933500 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.933802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.934582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.935218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.937144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.958538 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1348/2671] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:06:52 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:06:52.962763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:52.972986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01051f97-0087-47ef-83e0-73632c74aa20 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:52.977563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:52.978186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:52.978462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:52.978729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:52.979363 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:52.980938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:52.990656 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1324/2672] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:06:52.994517 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:53.004629 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c5142-3e52-4663-8925-9af3e17e7316 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:53.009181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:53.009792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:53.010070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:53.010338 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:53.010966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:53.013166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eacbe5-e140-4425-9ec5-ccc971c7b997 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:53.018399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:06:53.025031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1349/2673] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:06:53.030186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:53.044449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7057f77-00d8-45e4-8a56-5465e76e5bc8 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:53.049210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:53.049902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:53.050495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:53.050495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] PATH_INFO: `/v3/projects/fd2ff4aa388e449bbff4302f9b7096d9/users/7c86ce90f31a4d05b5edd9dea4545134/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:53.065643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2ff4aa388e449bbff4302f9b7096d9, user_id=7c86ce90f31a4d05b5edd9dea4545134, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:53.068364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:53.082360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1325/2674] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:06:53 2022] PUT /identity/v3/projects/fd2ff4aa388e449bbff4302f9b7096d9/users/7c86ce90f31a4d05b5edd9dea4545134/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:06:53.093518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:53.093841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:53.094121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:53.122792 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] MFA Rules not processed for user `7c86ce90f31a4d05b5edd9dea4545134`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:06:53.123933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:53.186160 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1350/2675] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:06:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:06:53.197413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:06:53.199099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:06:53.254843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865461db-ac7b-45bd-a73e-ccafba504361 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:06:53.258568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:06:53.259171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:06:53.259449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:06:53.259722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:06:53.268770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:06:53.270234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28219805-d7f3-4700-9e28-cf5cda61f394 service glance] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:06:53.278915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1326/2676] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:06:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:01.576656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:01.578118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:01.718408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144a61b-5105-4144-aee8-64e2a381574f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:01.722995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:01.723645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:01.723923 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:01.724193 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] PATH_INFO: `/v3/users/94ae3b9f39cd481c88c617e1e1463201` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:01.725331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ae3b9f39cd481c88c617e1e1463201)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:01.727271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:01.758886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:01.773676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ae3b9f39cd481c88c617e1e1463201'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:01.782419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ae3b9f39cd481c88c617e1e1463201'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:01.783034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ae3b9f39cd481c88c617e1e1463201'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:01.783634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ae3b9f39cd481c88c617e1e1463201'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:01.790827 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ae3b9f39cd481c88c617e1e1463201'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:01.807711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1351/2677] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:01 2022] DELETE /identity/v3/users/94ae3b9f39cd481c88c617e1e1463201 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:01.812111 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:01.812592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:01.812702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:01.840702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:01.841933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:01.898982 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1327/2678] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:01.909355 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:01.912311 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:01.992411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a4a72-9bc7-48ec-8a00-526f96e5c6f5 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:01.995991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:01.997127 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:01.997399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:01.997665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:02.006555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:02.007970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:02.017850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1352/2679] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:02.317007 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:02.318579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:02.331324 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eea79-26c4-4fd3-b2ca-d4d7a3c08079 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:02.335963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:02.336593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:02.336874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:02.337142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] PATH_INFO: `/v3/projects/4bd7c7df649140b296a9febd2ac937d7` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:02.338298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd7c7df649140b296a9febd2ac937d7)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:02.340887 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:02.378751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:02.390610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd7c7df649140b296a9febd2ac937d7', 'request_id': 'req-860579fb-571b-4538-92cd-a3fbef2b639c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:02.396885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd7c7df649140b296a9febd2ac937d7', 'request_id': 'req-860579fb-571b-4538-92cd-a3fbef2b639c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:02.404664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd7c7df649140b296a9febd2ac937d7', 'request_id': 'req-860579fb-571b-4538-92cd-a3fbef2b639c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:02.416886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] The token cache is being invalidate because project 4bd7c7df649140b296a9febd2ac937d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:02.417177 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:02.417831 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:02.419923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1328/2680] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:02 2022] DELETE /identity/v3/projects/4bd7c7df649140b296a9febd2ac937d7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:04.754955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:04.757399 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:04.860580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dda13-2200-45cf-a2bf-df1808265572 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:04.863759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:04.926982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:04.927603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:04.927887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:04.928159 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:04.929946 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:04.942548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:04.944124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:04.955552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1353/2681] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:07.206809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:07.208420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:07.299145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860579fb-571b-4538-92cd-a3fbef2b639c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:07.304166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:07.304771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:07.305055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:07.305327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:07.307074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:07.318258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:07.319847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec0284-d8e4-463e-a87b-000cf637fb07 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:07.330435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1329/2682] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:12.012712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.014141 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.077787 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd86d-b67d-45a2-b183-6eed973d0e86 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:12.082256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:12.082906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.083186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.083459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] PATH_INFO: `/v3/users/7c86ce90f31a4d05b5edd9dea4545134` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.084575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c86ce90f31a4d05b5edd9dea4545134)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:12.086515 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:12.133963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c86ce90f31a4d05b5edd9dea4545134'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.141512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c86ce90f31a4d05b5edd9dea4545134'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.142149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c86ce90f31a4d05b5edd9dea4545134'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.142748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c86ce90f31a4d05b5edd9dea4545134'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.149779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c86ce90f31a4d05b5edd9dea4545134'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.166488 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1354/2683] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:12 2022] DELETE /identity/v3/users/7c86ce90f31a4d05b5edd9dea4545134 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:12.171448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.171951 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.172075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.191082 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:12.192263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.249479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1330/2684] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:07:12.258978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.260501 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.274259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:12.320511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1966e2-a31e-4d01-99c2-4ba83e205c98 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:12.325096 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:12.325702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.326019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.326296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.335594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:12.336958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee22f4-4e4e-44bd-a861-31eab9ecf11c service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:12.346977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1355/2685] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:12.608586 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.610051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.622532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba4878-f8cc-43ae-8f17-57db960c79f4 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:12.627379 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:12.628197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.628478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.628762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] PATH_INFO: `/v3/projects/fd2ff4aa388e449bbff4302f9b7096d9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.629977 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2ff4aa388e449bbff4302f9b7096d9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:12.631909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:12.664472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:12.680489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2ff4aa388e449bbff4302f9b7096d9', 'request_id': 'req-aae826c9-9bb6-4f2e-840c-0f492a01e432'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.687284 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2ff4aa388e449bbff4302f9b7096d9', 'request_id': 'req-aae826c9-9bb6-4f2e-840c-0f492a01e432'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.694994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2ff4aa388e449bbff4302f9b7096d9', 'request_id': 'req-aae826c9-9bb6-4f2e-840c-0f492a01e432'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.705987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] The token cache is being invalidate because project fd2ff4aa388e449bbff4302f9b7096d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:12.706278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.706892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aae826c9-9bb6-4f2e-840c-0f492a01e432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:12.708340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1331/2686] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:12 2022] DELETE /identity/v3/projects/fd2ff4aa388e449bbff4302f9b7096d9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:07:12.713131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ce811-b45c-49ad-9372-f8337442160f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.713421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ce811-b45c-49ad-9372-f8337442160f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.713699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ce811-b45c-49ad-9372-f8337442160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.762310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-593ce811-b45c-49ad-9372-f8337442160f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:12.763523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-593ce811-b45c-49ad-9372-f8337442160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.820670 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1356/2687] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:12.828748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.829033 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.829304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.848768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:12.850010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:12.858346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1332/2688] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:12.862430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ce811-b45c-49ad-9372-f8337442160f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.872596 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ce811-b45c-49ad-9372-f8337442160f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:12.877309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:12.877940 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.878222 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.878493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.879129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:07:12.879430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:12.881026 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:12.892624 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1357/2689] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:07:12.896885 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.906105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e97c87-ff6f-4a1b-bc4a-25f6dd7f8301 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:12.910729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:12.911320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:12.911594 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:12.911865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:12.912503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:12.914440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:12.941401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1333/2690] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:12 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:12.945532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:12.955840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b63355-1be7-4d3c-b663-d095f0f661ac admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.010366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.010965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.011244 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.012051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.012687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.014660 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.036549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1358/2691] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:12 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:13.040751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.050824 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497e019-2af4-4d0e-8a71-c458dd5a7a4a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.055378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.055964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.056239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.056505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.057146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.058750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.068351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1334/2692] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:13.072259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.082551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8c6a5-a17b-4661-8402-30f111b9fb24 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.087129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.087708 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.087986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.088260 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.088964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.091199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde6d3c-5594-4f14-aadf-a9b37a83340e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.092398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:13.103049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1359/2693] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:07:13.106602 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.116471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2557e-ae78-4481-8e5d-5a12fb532b5e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.120967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.121601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.121914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.122187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] PATH_INFO: `/v3/projects/78028c571a5547d68958e16a99fb0e3d/users/9e4a7b8a6fef4e73aac351319bf37b2a/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.136855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78028c571a5547d68958e16a99fb0e3d, user_id=9e4a7b8a6fef4e73aac351319bf37b2a, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.139526 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.151895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1335/2694] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:13 2022] PUT /identity/v3/projects/78028c571a5547d68958e16a99fb0e3d/users/9e4a7b8a6fef4e73aac351319bf37b2a/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:13.163745 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.164029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.164328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.191635 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] MFA Rules not processed for user `9e4a7b8a6fef4e73aac351319bf37b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:13.192832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:13.255414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1360/2695] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:07:13.260002 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.320314 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d621821-94b1-4882-82aa-a216e2d3a966 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.325832 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.326430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.326740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.327015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.327642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.329541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.354921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1336/2696] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:13.359042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.369784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dad975-57a1-45fc-93e2-5eef41257634 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.424577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.425172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.425446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.425717 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.426378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.428273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.450051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1361/2697] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:07:13.453995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.464445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f124b3-0fb6-4a25-9217-f4a8f089325a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.469003 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.469599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.469923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.470202 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.470843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.472430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.482915 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1337/2698] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:13.486625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.496875 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a4b96-8956-4d61-b195-16b505be5c8f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.501397 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.502012 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.502289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.502558 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.503189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.504753 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996db83-b092-4c3b-a813-af426fdae166 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.514884 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1362/2699] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:13.518878 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.528693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c2aa1-09de-4fa9-a670-93365eed15c9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.533364 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.534060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.534341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.534612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] PATH_INFO: `/v3/projects/0ca479605d0c4312972a5ef4d48bc03f/users/d223953975bf46aabf3c89e6088c0e03/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.549890 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca479605d0c4312972a5ef4d48bc03f, user_id=d223953975bf46aabf3c89e6088c0e03, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.552507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.565385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1338/2700] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:13 2022] PUT /identity/v3/projects/0ca479605d0c4312972a5ef4d48bc03f/users/d223953975bf46aabf3c89e6088c0e03/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:13.575770 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.576112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.576388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.603595 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] MFA Rules not processed for user `d223953975bf46aabf3c89e6088c0e03`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:13.604807 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:13.658420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:13.669878 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1363/2701] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:13.674573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.679011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:13.737644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179c5bd-004a-4073-8b8d-dac1b908bd9e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.742243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.742838 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.743112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.743380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.744031 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.746585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.773697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1339/2702] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:13.777789 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.788608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27377fc5-d909-4b25-88d7-deb0af281f8b None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.845590 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.846231 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.846534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.846795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.847423 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.849360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.870147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1364/2703] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:13 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:13.874771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.884928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d266-9ea9-4ad7-ac52-f1a340db3cf9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.889900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.890124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.890419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.890669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.891301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.892898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.904465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1340/2704] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:07:13.908630 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.919000 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4fbe9-e94e-47e7-b985-d33ce4f46ad3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:13.923557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:13.924223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:13.924498 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:13.924766 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] PATH_INFO: `/v3/projects/2028b2bca2f345368f0634411b5ad1ce/users/01f80d2b9f1c47e98a6b761d1f2f355a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:13.938976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2028b2bca2f345368f0634411b5ad1ce, user_id=01f80d2b9f1c47e98a6b761d1f2f355a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:13.941758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:13.955162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1365/2705] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:13 2022] PUT /identity/v3/projects/2028b2bca2f345368f0634411b5ad1ce/users/01f80d2b9f1c47e98a6b761d1f2f355a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:13.958921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:13.969050 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6e05-dec7-4e36-b0aa-bdca755102b2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:14.023124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:14.023713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:14.023987 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:14.024256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:14.024880 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:14.026482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9768a-6338-465e-82a8-7cd627e65e8b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:14.036554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1341/2706] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:14.040701 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:14.051722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23140ad-8671-4f30-8aa9-21e5b15fcd02 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:14.056230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:14.056869 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:14.057148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:14.057414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] PATH_INFO: `/v3/projects/2028b2bca2f345368f0634411b5ad1ce/users/01f80d2b9f1c47e98a6b761d1f2f355a/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:14.059468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2028b2bca2f345368f0634411b5ad1ce, user_id=01f80d2b9f1c47e98a6b761d1f2f355a, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:14.062108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:14.071936 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1366/2707] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:14 2022] PUT /identity/v3/projects/2028b2bca2f345368f0634411b5ad1ce/users/01f80d2b9f1c47e98a6b761d1f2f355a/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:14.081958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:14.082237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:14.082507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:14.110251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] MFA Rules not processed for user `01f80d2b9f1c47e98a6b761d1f2f355a`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:14.111424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:14.171906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:14.178254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1342/2708] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:14.189413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:14.191163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:14.248759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9353b6-5537-4b62-9f97-04aa647a954a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:14.252334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:14.252920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:14.253195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:14.253462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:14.307941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:14.309366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:14.313882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:14.318625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1367/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:15.820729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:15.822424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:15.835665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730f32b-be13-49f1-b489-630539eb1cb6 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:15.839232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:15.840415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:15.840696 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:15.840981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:15.897055 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:15.898531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:15.907917 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1343/2710] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:16.415858 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:16.427003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f65-98a6-475c-88d3-0cd691703072 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:16.430769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:16.431378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:16.431653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:16.431922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:16.443359 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:16.445651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30c243-0d10-4cf9-954f-a75c84f798b2 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:16.461099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1368/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:20.321439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:20.382707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff97ac-3ae4-4854-8e7a-c4ef92228001 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:20.387718 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:20.388335 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:20.388619 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:20.388892 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] PATH_INFO: `/v3/users/9e4a7b8a6fef4e73aac351319bf37b2a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:20.390043 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4a7b8a6fef4e73aac351319bf37b2a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:20.391962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:20.436051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4a7b8a6fef4e73aac351319bf37b2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:20.443212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4a7b8a6fef4e73aac351319bf37b2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:20.443827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4a7b8a6fef4e73aac351319bf37b2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:20.444428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4a7b8a6fef4e73aac351319bf37b2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:20.452105 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4a7b8a6fef4e73aac351319bf37b2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:20.469756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1344/2712] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:20 2022] DELETE /identity/v3/users/9e4a7b8a6fef4e73aac351319bf37b2a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:20.474240 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:20.474726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:20.474844 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:20.493082 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:20.494306 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:20.552941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1369/2713] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:07:20.562740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:20.564136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:20.583755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:20.644293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d4f7-d056-4a77-900c-e79b84cec012 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:20.647929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:20.648533 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:20.648811 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:20.649081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:20.658781 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:20.660492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:20.670330 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1345/2714] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:20.933443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:20.934943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:20.948185 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd9ba8-f8d7-4b8a-8166-de96e99d80d5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:20.953357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:20.954035 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:20.954313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:20.954584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] PATH_INFO: `/v3/projects/78028c571a5547d68958e16a99fb0e3d` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:20.955699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78028c571a5547d68958e16a99fb0e3d)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:20.957615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:21.003998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78028c571a5547d68958e16a99fb0e3d', 'request_id': 'req-ae31a13e-2567-4204-a9f3-f434b38734e6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.010181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78028c571a5547d68958e16a99fb0e3d', 'request_id': 'req-ae31a13e-2567-4204-a9f3-f434b38734e6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.017916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78028c571a5547d68958e16a99fb0e3d', 'request_id': 'req-ae31a13e-2567-4204-a9f3-f434b38734e6'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.028174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] The token cache is being invalidate because project 78028c571a5547d68958e16a99fb0e3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:21.028471 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.029075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.030594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1370/2715] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:20 2022] DELETE /identity/v3/projects/78028c571a5547d68958e16a99fb0e3d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:21.034553 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:21.044648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccb6f-b5e9-4773-a205-f6b764f7827a service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:21.101334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:21.102016 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:21.102290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:21.102557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] PATH_INFO: `/v3/users/d223953975bf46aabf3c89e6088c0e03` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:21.103651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d223953975bf46aabf3c89e6088c0e03)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:21.106226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:21.151040 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd223953975bf46aabf3c89e6088c0e03'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.158075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd223953975bf46aabf3c89e6088c0e03'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.158673 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd223953975bf46aabf3c89e6088c0e03'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.159265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd223953975bf46aabf3c89e6088c0e03'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.166546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd223953975bf46aabf3c89e6088c0e03'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.184088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1346/2716] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:21 2022] DELETE /identity/v3/users/d223953975bf46aabf3c89e6088c0e03 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:21.466652 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:21.468098 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:21.481569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a13e-2567-4204-a9f3-f434b38734e6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:21.536295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:21.536847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:21.537119 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:21.537386 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] PATH_INFO: `/v3/projects/0ca479605d0c4312972a5ef4d48bc03f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:21.538508 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca479605d0c4312972a5ef4d48bc03f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:21.540390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:21.552029 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:21.586918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca479605d0c4312972a5ef4d48bc03f', 'request_id': 'req-4be1f431-2459-4d26-af51-ef3b5ecf96ac'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.593418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca479605d0c4312972a5ef4d48bc03f', 'request_id': 'req-4be1f431-2459-4d26-af51-ef3b5ecf96ac'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.600758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca479605d0c4312972a5ef4d48bc03f', 'request_id': 'req-4be1f431-2459-4d26-af51-ef3b5ecf96ac'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.612066 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] The token cache is being invalidate because project 0ca479605d0c4312972a5ef4d48bc03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:21.612351 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.612957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.614997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1371/2717] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:21 2022] DELETE /identity/v3/projects/0ca479605d0c4312972a5ef4d48bc03f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:21.618909 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:21.620415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:21.632752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9b728-b87e-4302-9feb-45dd2d6027fc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:21.645085 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:21.691562 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:21.692187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:21.692464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:21.692753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] PATH_INFO: `/v3/users/01f80d2b9f1c47e98a6b761d1f2f355a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:21.693883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f80d2b9f1c47e98a6b761d1f2f355a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:21.695800 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:21.743294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f80d2b9f1c47e98a6b761d1f2f355a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.750368 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f80d2b9f1c47e98a6b761d1f2f355a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.750976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f80d2b9f1c47e98a6b761d1f2f355a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.751630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f80d2b9f1c47e98a6b761d1f2f355a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.758540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e2a02ce1-74e7-4b63-b4fa-f5d1e74332ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f80d2b9f1c47e98a6b761d1f2f355a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:21.775286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1347/2718] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:21 2022] DELETE /identity/v3/users/01f80d2b9f1c47e98a6b761d1f2f355a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:22.053396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.054930 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:22.068059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1f431-2459-4d26-af51-ef3b5ecf96ac admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.127767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.128472 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.128747 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.129039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] PATH_INFO: `/v3/projects/2028b2bca2f345368f0634411b5ad1ce` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.130405 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2028b2bca2f345368f0634411b5ad1ce)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.132334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.178506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2028b2bca2f345368f0634411b5ad1ce', 'request_id': 'req-317bf16c-75d4-4152-8926-558bd8bdebcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:22.184667 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2028b2bca2f345368f0634411b5ad1ce', 'request_id': 'req-317bf16c-75d4-4152-8926-558bd8bdebcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:22.191942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2028b2bca2f345368f0634411b5ad1ce', 'request_id': 'req-317bf16c-75d4-4152-8926-558bd8bdebcf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:22.202430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] The token cache is being invalidate because project 2028b2bca2f345368f0634411b5ad1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:22.202712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:22.203349 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-317bf16c-75d4-4152-8926-558bd8bdebcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:22.204794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1372/2719] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:22 2022] DELETE /identity/v3/projects/2028b2bca2f345368f0634411b5ad1ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:22.209643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.209964 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.210240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.229628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:22.231428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:22.288830 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1348/2720] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:22.351603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.351851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.352057 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.371974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:22.373195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:22.380944 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1373/2721] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:22.385000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.394560 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487b466-a086-47cd-b1d2-ee3db92436b9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.399131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.399747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.400020 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.400283 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.400919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:07:22.401217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.403374 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.414523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1349/2722] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:07:22.419108 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.427984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb7d5-3781-4be7-ac67-fc15072d73c5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.432573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.433156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.433430 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.433694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.434354 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.436259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.463049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1374/2723] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:07:22.467723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.477585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258e6a-736a-4cfe-9b7d-7ad765cc9b37 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.499341 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:22.539100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.539692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.539968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.540240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.540861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.543431 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.563233 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1350/2724] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:22.567371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.573204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:22.580418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71299ef-35d2-4085-b502-569199137a15 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.585894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.586495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.586774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.587047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.587688 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.589924 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.599750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1375/2725] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:22.603710 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.613289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9d34c-fd04-447e-8a25-899a7bc10e3c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.617858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.618445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.618723 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.618995 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.619630 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.621241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278f859-6e02-4e34-a978-cf547796c1e8 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.631531 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1351/2726] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:22.635228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.645366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f396-831a-4869-9e96-d35b6466866c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.649977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.650863 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.651172 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.651448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] PATH_INFO: `/v3/projects/05f12b088b62469fbfdbd39c35efac00/users/62d98071ebdb4b72a32ac2fc22ac6f4f/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.667391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f12b088b62469fbfdbd39c35efac00, user_id=62d98071ebdb4b72a32ac2fc22ac6f4f, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.670065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.682414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1376/2727] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:22 2022] PUT /identity/v3/projects/05f12b088b62469fbfdbd39c35efac00/users/62d98071ebdb4b72a32ac2fc22ac6f4f/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:07:22.692729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.693018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.693286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.721507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] MFA Rules not processed for user `62d98071ebdb4b72a32ac2fc22ac6f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:22.722768 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:22.785035 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1352/2728] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:22.789710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.850817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feb1b7-75a5-4994-9042-967f19c4659d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.855680 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.856251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.856534 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.856815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.857457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.859425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.885634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1377/2729] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:22.890489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.901424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f6721-55c4-4bc8-97dd-64b32c9fbe60 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.957210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:22.957834 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:22.958110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:22.958378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:22.958993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:22.960888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:22.980437 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1353/2730] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:22 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:22.984554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:22.994604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460049c-0fda-47a1-b8cc-f343c082aef1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:22.999890 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.000487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.000764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.001038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.001678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.003309 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.014285 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1378/2731] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:23.018076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.027872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabef9c3-2b15-4534-9119-50622d9090dc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.032444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.033029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.033305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.033580 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.034250 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.035825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dc9e2-8329-4012-acde-06fa57b9270a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.045669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1354/2732] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:23.049548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.060480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a5288-8ff2-4ed6-a689-de4651f02809 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.065179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.065857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.066142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.066416 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] PATH_INFO: `/v3/projects/3e59bb7a33204c6a95c906ff161e44bb/users/7c13a4902d5c49b18659bae602da4842/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.081719 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e59bb7a33204c6a95c906ff161e44bb, user_id=7c13a4902d5c49b18659bae602da4842, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.084529 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.098297 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1379/2733] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:23 2022] PUT /identity/v3/projects/3e59bb7a33204c6a95c906ff161e44bb/users/7c13a4902d5c49b18659bae602da4842/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:23.109558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.109798 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.110104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.130669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:23.139188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] MFA Rules not processed for user `7c13a4902d5c49b18659bae602da4842`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:23.140584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:23.204564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1355/2734] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:23.209042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.270321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d91c86-1bf9-427a-9f92-e4318cbb6edc admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.275039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103228-47b5-4726-8304-773b70739219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.275646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103228-47b5-4726-8304-773b70739219 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.275950 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103228-47b5-4726-8304-773b70739219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.276226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30103228-47b5-4726-8304-773b70739219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.276912 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30103228-47b5-4726-8304-773b70739219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.278855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30103228-47b5-4726-8304-773b70739219 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.304929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1380/2735] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:23 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:23.309844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.320539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f433d0-b553-47db-b4eb-a0733fa42e8a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.377259 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.377844 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.378164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.378459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.379123 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.381114 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.401946 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1356/2736] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:23 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:07:23.405855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103228-47b5-4726-8304-773b70739219 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.411251 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:23.418509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30103228-47b5-4726-8304-773b70739219 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.423110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.423730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.424007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.424277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.424917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.426557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.437411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1381/2737] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:23.441543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.452235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4ae45-e30d-48af-adc9-801b49d7fa94 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.457030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.457672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.457985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.458261 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] PATH_INFO: `/v3/projects/f6dff499472b4c28b7962e193e8264c3/users/0b783479c0ff41dbb7b1b380103a56be/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.472810 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dff499472b4c28b7962e193e8264c3, user_id=0b783479c0ff41dbb7b1b380103a56be, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.475546 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.488543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1357/2738] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:23 2022] PUT /identity/v3/projects/f6dff499472b4c28b7962e193e8264c3/users/0b783479c0ff41dbb7b1b380103a56be/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:23.492340 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.502822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b089fa-a7a3-4903-a0bf-889ece47dc2d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.556642 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.557238 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.557516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.557895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.559074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.560826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37ef9-f9c8-40c5-8416-d7a8db1ed297 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.570318 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1382/2739] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:23.574697 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.584738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90e34-38b4-4ac4-9907-948cdde0c48e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.589345 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.590015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.590294 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.590570 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] PATH_INFO: `/v3/projects/f6dff499472b4c28b7962e193e8264c3/users/0b783479c0ff41dbb7b1b380103a56be/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.592458 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dff499472b4c28b7962e193e8264c3, user_id=0b783479c0ff41dbb7b1b380103a56be, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.595124 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.606168 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1358/2740] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:23 2022] PUT /identity/v3/projects/f6dff499472b4c28b7962e193e8264c3/users/0b783479c0ff41dbb7b1b380103a56be/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:07:23.615972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.616277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.616550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.645521 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] MFA Rules not processed for user `0b783479c0ff41dbb7b1b380103a56be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:23.646676 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:23.712796 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1383/2741] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:23.724360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:23.726136 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:23.783100 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:23.805125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bb2b3-dd08-4578-a07d-64f60da6c98f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:23.808753 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:23.809357 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:23.809635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:23.809942 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:23.864971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:23.866430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:23.875029 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1359/2742] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:25.368694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:25.370143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:25.381903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020f8b2-38ae-4b36-8eb9-6082a552d4cd None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:25.385382 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:25.386017 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:25.386296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:25.386565 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:25.442518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:25.443870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:25.452927 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1384/2743] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:25.971048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:25.979612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526015c7-73ca-45b2-84d2-01b7800d3557 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:25.983278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:25.983867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:25.984142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:25.984411 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:25.993133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:25.994642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba66bd6-ba16-4a0f-aac4-8f744def6833 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:26.004338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1360/2744] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:29.856156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:29.868170 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:29.918730 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdf2a4-128e-4f77-85d9-95bc208e4446 service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:29.923312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:29.923933 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:29.924209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:29.924479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] PATH_INFO: `/v3/users/62d98071ebdb4b72a32ac2fc22ac6f4f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:29.925606 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d98071ebdb4b72a32ac2fc22ac6f4f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:29.927528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:29.972476 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d98071ebdb4b72a32ac2fc22ac6f4f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:29.979332 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d98071ebdb4b72a32ac2fc22ac6f4f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:29.980065 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d98071ebdb4b72a32ac2fc22ac6f4f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:29.980662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d98071ebdb4b72a32ac2fc22ac6f4f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:29.987547 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d98071ebdb4b72a32ac2fc22ac6f4f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.005337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1385/2745] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:29 2022] DELETE /identity/v3/users/62d98071ebdb4b72a32ac2fc22ac6f4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:30.009853 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:30.010323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:30.010447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:30.029475 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:30.030713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:30.088322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1361/2746] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:30.097834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:30.099221 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:30.156816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5561917-0f22-46a3-833f-7229e1af3fce admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:30.160411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:30.161007 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:30.161280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:30.161545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:30.170874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:30.172298 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:30.181815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1386/2747] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:30.437014 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:30.438435 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:30.456402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab6b39-f206-426e-84bb-decff8fe7c3a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:30.461266 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:30.461937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:30.462210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:30.462509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] PATH_INFO: `/v3/projects/05f12b088b62469fbfdbd39c35efac00` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:30.464388 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f12b088b62469fbfdbd39c35efac00)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:30.466565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:30.512985 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f12b088b62469fbfdbd39c35efac00', 'request_id': 'req-1a929eaf-0a75-4a5f-a11d-1e451930df98'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.519197 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f12b088b62469fbfdbd39c35efac00', 'request_id': 'req-1a929eaf-0a75-4a5f-a11d-1e451930df98'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.528342 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f12b088b62469fbfdbd39c35efac00', 'request_id': 'req-1a929eaf-0a75-4a5f-a11d-1e451930df98'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.535155 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:30.541756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] The token cache is being invalidate because project 05f12b088b62469fbfdbd39c35efac00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:30.542817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.543691 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.545146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1362/2748] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:30 2022] DELETE /identity/v3/projects/05f12b088b62469fbfdbd39c35efac00 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:30.549364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:30.559241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9fba0-709d-4950-8977-48dbe4e43fe4 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:30.589637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:30.617649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:30.618237 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:30.618511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:30.618780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] PATH_INFO: `/v3/users/7c13a4902d5c49b18659bae602da4842` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:30.619876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c13a4902d5c49b18659bae602da4842)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:30.621972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:30.665622 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c13a4902d5c49b18659bae602da4842'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.673836 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c13a4902d5c49b18659bae602da4842'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.674451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c13a4902d5c49b18659bae602da4842'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.675050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c13a4902d5c49b18659bae602da4842'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.681980 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c13a4902d5c49b18659bae602da4842'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:30.699671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1387/2749] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:30 2022] DELETE /identity/v3/users/7c13a4902d5c49b18659bae602da4842 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:30.978012 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:30.979590 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:30.993258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a929eaf-0a75-4a5f-a11d-1e451930df98 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.050703 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.051295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.051532 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.051806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] PATH_INFO: `/v3/projects/3e59bb7a33204c6a95c906ff161e44bb` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.052938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e59bb7a33204c6a95c906ff161e44bb)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.054900 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:31.102350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e59bb7a33204c6a95c906ff161e44bb', 'request_id': 'req-9e03151a-fa65-4e4d-8494-f931290877d1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.108425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e59bb7a33204c6a95c906ff161e44bb', 'request_id': 'req-9e03151a-fa65-4e4d-8494-f931290877d1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.115015 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e59bb7a33204c6a95c906ff161e44bb', 'request_id': 'req-9e03151a-fa65-4e4d-8494-f931290877d1'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.125332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] The token cache is being invalidate because project 3e59bb7a33204c6a95c906ff161e44bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:31.125618 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.126267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.127729 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1363/2750] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:30 2022] DELETE /identity/v3/projects/3e59bb7a33204c6a95c906ff161e44bb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:31.131736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:31.133236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:31.147286 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e281424-959b-4e28-b6ed-4548b6532dbd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.203162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.203785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.204064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.204341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] PATH_INFO: `/v3/users/0b783479c0ff41dbb7b1b380103a56be` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.205483 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b783479c0ff41dbb7b1b380103a56be)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.207418 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:31.251971 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b783479c0ff41dbb7b1b380103a56be'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.259059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b783479c0ff41dbb7b1b380103a56be'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.259750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b783479c0ff41dbb7b1b380103a56be'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.260401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b783479c0ff41dbb7b1b380103a56be'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.262216 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:31.270304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-2649381e-1572-441f-a3dd-fae532aedea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b783479c0ff41dbb7b1b380103a56be'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.287312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1388/2751] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:31 2022] DELETE /identity/v3/users/0b783479c0ff41dbb7b1b380103a56be => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:07:31.554954 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:31.556402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:31.569808 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03151a-fa65-4e4d-8494-f931290877d1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.625641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.626308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.626582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.626850 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] PATH_INFO: `/v3/projects/f6dff499472b4c28b7962e193e8264c3` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.627958 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6dff499472b4c28b7962e193e8264c3)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.629889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:31.662013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:31.676740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6dff499472b4c28b7962e193e8264c3', 'request_id': 'req-f56869a5-0902-4571-80be-16d92bfc1d86'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.683569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6dff499472b4c28b7962e193e8264c3', 'request_id': 'req-f56869a5-0902-4571-80be-16d92bfc1d86'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.692237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6dff499472b4c28b7962e193e8264c3', 'request_id': 'req-f56869a5-0902-4571-80be-16d92bfc1d86'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.703220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] The token cache is being invalidate because project f6dff499472b4c28b7962e193e8264c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:31.703509 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.704144 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f56869a5-0902-4571-80be-16d92bfc1d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:31.705611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1364/2752] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:31 2022] DELETE /identity/v3/projects/f6dff499472b4c28b7962e193e8264c3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:31.711150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.711438 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.711716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.731352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:31.732541 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:31.791406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1389/2753] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:31.800460 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.800863 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.801191 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.823714 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:31.824920 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:31.832648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1365/2754] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:07:31.837052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:31.848006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20d2a-0048-4ed5-be69-bcec2f1dd6f1 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.853906 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.855033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.855336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.855605 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.856257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:07:31.856559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.858204 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:31.872128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1390/2755] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:07:31.876440 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:31.885814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f57d2c-0291-4445-9719-8a50d6635412 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.890635 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.891301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.891585 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.891857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.892495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.894463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:31.922882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1366/2756] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:07:31 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:31.927144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:31.937120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c96314-3b34-456a-9dd5-5acc04f7a4b6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:31.993112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:31.993714 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:31.994018 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:31.994290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:31.994931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:31.996832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:32.017050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1391/2757] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:07:31 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:07:32.021113 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:32.031965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441a393-ae3d-433b-8fd7-aaa565ccb1ef admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:32.036572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:32.037165 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:32.037439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:32.037704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:32.038376 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:32.039959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:32.049762 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1367/2758] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:32.053550 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:32.063943 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1793d-5338-4018-99e1-4423db383f54 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:32.068478 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:32.069058 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:32.069336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:32.069604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:32.070270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:32.071847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28aafe-4ece-4195-9041-60d86266f3ab admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:32.082391 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1392/2759] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:07:32.086428 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:32.096948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f250cf-ea37-47d7-96f9-ab9aabda8dda admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:32.101555 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:32.102239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:32.102520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:32.102794 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] PATH_INFO: `/v3/projects/8712d8f54ec24b9c8bf008afe56b8d25/users/563183787ab145b691240f9599118daf/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:32.117700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8712d8f54ec24b9c8bf008afe56b8d25, user_id=563183787ab145b691240f9599118daf, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:32.120362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:32.134175 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1368/2760] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:07:32 2022] PUT /identity/v3/projects/8712d8f54ec24b9c8bf008afe56b8d25/users/563183787ab145b691240f9599118daf/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:32.145636 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:32.146175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:32.146278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:32.174532 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] MFA Rules not processed for user `563183787ab145b691240f9599118daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:32.175760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:32.202591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:32.242174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1393/2761] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:32.253090 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:32.254867 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:32.311039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150edb0b-26d3-46ca-bf0e-a857eba49beb admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:32.314675 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:32.315297 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:32.315574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:32.315843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:32.324750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:32.326216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cec02-626e-4cfa-9c56-72d2d5bcfc62 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:32.334739 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1369/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:34.875684 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:34.936928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d2fe-6fe0-4af5-b8b8-8351f3513c13 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:34.941390 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:34.942038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:34.942312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:34.942578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] PATH_INFO: `/v3/users/563183787ab145b691240f9599118daf` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:34.943672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563183787ab145b691240f9599118daf)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:34.945570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:34.991070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563183787ab145b691240f9599118daf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:34.998348 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563183787ab145b691240f9599118daf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:34.999042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563183787ab145b691240f9599118daf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:34.999649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563183787ab145b691240f9599118daf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.006473 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563183787ab145b691240f9599118daf'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.023665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1394/2763] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:07:34 2022] DELETE /identity/v3/users/563183787ab145b691240f9599118daf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:07:35.028097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:35.028564 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:35.028664 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:35.048186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:07:35.049366 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:35.099227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:35.109793 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1370/2764] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:07:35.119366 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:35.121301 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:35.171982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:35.178278 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0b58a-f379-4b43-beb0-369f285fda56 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:35.183765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:35.184372 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:35.184648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:35.184915 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:35.193922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:35.195335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a619119-7126-4e89-a4d1-e6655740aafc service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:35.205626 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1395/2765] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:07:35.466628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:07:35.468139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:07:35.480520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9a963-0263-4d0c-8afa-7d9a8b5bc364 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:07:35.486274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:07:35.487212 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:07:35.487633 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:07:35.487903 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] PATH_INFO: `/v3/projects/8712d8f54ec24b9c8bf008afe56b8d25` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:07:35.489581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8712d8f54ec24b9c8bf008afe56b8d25)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:07:35.492390 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:07:35.537827 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8712d8f54ec24b9c8bf008afe56b8d25', 'request_id': 'req-6fb178d6-f61d-488e-87f8-ac3a30094abc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.544681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8712d8f54ec24b9c8bf008afe56b8d25', 'request_id': 'req-6fb178d6-f61d-488e-87f8-ac3a30094abc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.551491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8712d8f54ec24b9c8bf008afe56b8d25', 'request_id': 'req-6fb178d6-f61d-488e-87f8-ac3a30094abc'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.563750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] The token cache is being invalidate because project 8712d8f54ec24b9c8bf008afe56b8d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:07:35.564037 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.564665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:07:35.566174 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1371/2766] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:07:35 2022] DELETE /identity/v3/projects/8712d8f54ec24b9c8bf008afe56b8d25 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:07:45.201530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=5 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:07:45.560638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:00.853190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cff2b-50da-4a71-ba0e-68ba346ecc09 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:00.853693 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cff2b-50da-4a71-ba0e-68ba346ecc09 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:00.853829 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cff2b-50da-4a71-ba0e-68ba346ecc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:00.884852 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-d28cff2b-50da-4a71-ba0e-68ba346ecc09 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:08:00.886089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-d28cff2b-50da-4a71-ba0e-68ba346ecc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:00.942326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1396/2767] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:08:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:08:00.987296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:00.988704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:01.058348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb178d6-f61d-488e-87f8-ac3a30094abc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:01.064568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:01.065315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:01.065759 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:01.066219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:01.075745 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:01.077339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e6d76-0a65-476a-8234-75a88141111a service swift] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:01.085789 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1372/2768] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:08:10.935694 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=29 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:11.082521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=32 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:14.867645 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:14.868229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:14.868229 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] PATH_INFO: `/` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:14.869958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1397/2769] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 08:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 08:08:14.875298 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecac1-3000-48f4-b5e9-43bfef17d85b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:14.875814 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecac1-3000-48f4-b5e9-43bfef17d85b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:14.876186 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecac1-3000-48f4-b5e9-43bfef17d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:14.913147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-d17ecac1-3000-48f4-b5e9-43bfef17d85b None None] MFA Rules not processed for user `742f83bd38b542d193a911ef623abd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:08:14.914755 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d17ecac1-3000-48f4-b5e9-43bfef17d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:15.019161 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1373/2770] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 08:08:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:08:15.030494 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:15.032219 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:15.095189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9db98d-94d2-44e6-9b7b-92f606a1f4ed None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:15.098815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ec0de2bbadf4dbd9a5b66992b382c17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:15.099427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:15.099704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:15.099976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:15.111050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:15.112486 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:15.124077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1398/2771] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:08:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:08:17.565169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:17.565672 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:17.565796 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:17.585849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:08:17.587054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:17.594955 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1374/2772] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 08:08:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:08:17.606928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:17.618128 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d816437-3c3c-43c6-a315-2cd2ed4d0fe2 service glance] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:17.623528 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:17.624112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:17.624388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:17.624656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:17.632445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:17.634048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:17.642398 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1399/2773] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:08:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:08:24.602854 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:24.604938 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:24.747274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a52054-77cc-4e34-840e-84c9ccacb51f None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:24.759728 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:24.760536 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:24.761450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:24.762048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:24.764679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:24.987030 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:24.989350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:25.015247 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1375/2774] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:08:26.756053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:26.757536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:26.815614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:26.817234 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:26.834036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1d98f-f852-4872-92e3-d4a00a5ce07e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:26.838668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:26.839859 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:26.840140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:26.840803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] PATH_INFO: `/v3/users/f07e6a2e509e4aba865fcd6e935afa2a` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:26.841981 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07e6a2e509e4aba865fcd6e935afa2a)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:26.842883 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bb2fe-ab72-4710-bab6-673f759aee55 service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:26.843975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:26.848118 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:26.848880 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:26.849186 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:26.849457 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:26.851228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:26.863916 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:26.865617 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:26.879099 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1400/2775] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:08:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:08:26.882289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:26.909993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07e6a2e509e4aba865fcd6e935afa2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:26.918520 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07e6a2e509e4aba865fcd6e935afa2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:26.919166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07e6a2e509e4aba865fcd6e935afa2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:26.919791 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07e6a2e509e4aba865fcd6e935afa2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:26.927824 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07e6a2e509e4aba865fcd6e935afa2a'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:26.945402 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1376/2776] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:08:26 2022] DELETE /identity/v3/users/f07e6a2e509e4aba865fcd6e935afa2a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:08:27.110811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:27.123458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac75901-f7fa-41d4-a240-58902e6658fd service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:27.180370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:27.180992 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:27.181267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:27.181536 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] PATH_INFO: `/v3/projects/f7a64886550f4140bf90a77070f769e0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:27.182695 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a64886550f4140bf90a77070f769e0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:27.184583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:27.207640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:27.236039 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a64886550f4140bf90a77070f769e0', 'request_id': 'req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.245509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a64886550f4140bf90a77070f769e0', 'request_id': 'req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.255326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a64886550f4140bf90a77070f769e0', 'request_id': 'req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.269615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] The token cache is being invalidate because project f7a64886550f4140bf90a77070f769e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:08:27.269968 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.270598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.272809 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1401/2777] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:08:27 2022] DELETE /identity/v3/projects/f7a64886550f4140bf90a77070f769e0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:08:27.278225 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:27.280065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:27.299599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191954a-522f-4f21-b46e-b1c37a9054de admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:27.356609 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:27.357246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:27.357515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:27.357816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] PATH_INFO: `/v3/users/985634f35e8d42a78ca53eac3924fb35` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:27.358975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985634f35e8d42a78ca53eac3924fb35)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:27.360898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:27.411473 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985634f35e8d42a78ca53eac3924fb35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.419080 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985634f35e8d42a78ca53eac3924fb35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.419812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985634f35e8d42a78ca53eac3924fb35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.420441 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985634f35e8d42a78ca53eac3924fb35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.427548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-72f3b675-62a4-4268-aa08-291f649b3c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985634f35e8d42a78ca53eac3924fb35'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.447419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1377/2778] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:08:27 2022] DELETE /identity/v3/users/985634f35e8d42a78ca53eac3924fb35 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:08:27.610960 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:27.612526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:27.627328 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6dbb9f-0dc6-4ee0-9fb8-2d8cb670604b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:27.693591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:27.695182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:27.695182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:27.695182 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] PATH_INFO: `/v3/projects/01408158bb0345bf9b8f5ff547c84bcb` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:27.696422 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] RBAC: Authorizing `identity:delete_project(project_id=01408158bb0345bf9b8f5ff547c84bcb)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:27.700168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:27.746551 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01408158bb0345bf9b8f5ff547c84bcb', 'request_id': 'req-c6095a1c-3db5-4587-b6f3-301b052c071f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.752917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01408158bb0345bf9b8f5ff547c84bcb', 'request_id': 'req-c6095a1c-3db5-4587-b6f3-301b052c071f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.760101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01408158bb0345bf9b8f5ff547c84bcb', 'request_id': 'req-c6095a1c-3db5-4587-b6f3-301b052c071f'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.770442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] The token cache is being invalidate because project 01408158bb0345bf9b8f5ff547c84bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:08:27.770734 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.771360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:08:27.772820 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1402/2779] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:08:27 2022] DELETE /identity/v3/projects/01408158bb0345bf9b8f5ff547c84bcb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:08:32.685318 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:32.685855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:32.685976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:32.706610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] MFA Rules not processed for user `6555dc699d9d434eb866eaf08805ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:08:32.707809 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:32.728088 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:32.763554 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1378/2780] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 08:08:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:08:32.775755 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:08:32.777671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:08:32.820725 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:32.850570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6095a1c-3db5-4587-b6f3-301b052c071f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:08:32.856535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89e32c33e2184f5c93cc526f085c5375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:08:32.857139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:08:32.857439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:08:32.857712 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:08:32.865832 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:08:32.867414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:08:32.875447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1403/2781] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 08:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 08:08:42.756941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=13 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:08:42.872116 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=14 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:03.247966 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:03.249352 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:03.304403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f312-ca8c-4936-9062-bcb5c3d702a7 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:03.307948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:03.308541 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:03.308812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:03.309074 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:03.310771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:03.372333 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:03.373770 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:03.382557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1379/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:03.713859 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:03.715321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:03.770583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709594ef-6042-464d-9ca8-d4285474427b service swift] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:03.774060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:03.774644 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:03.774920 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:03.775190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:03.776881 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:03.847444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:03.848870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:03.859578 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1404/2783] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:05.045485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:05.047131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:05.058599 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136137e1-372b-4c5b-9053-b76f94855b2d service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:05.062348 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:05.062965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:05.063241 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:05.063507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:05.065244 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:05.078926 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:05.080346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:05.092240 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1380/2784] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:11.516167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:11.524741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65c917-7f67-4085-9d41-1f9b95d31b5b service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:11.528962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:11.529555 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:11.529871 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:11.530171 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:11.531854 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:11.598163 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:11.599600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:11.611758 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1405/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:14.808592 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:14.810096 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:14.824512 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41294c00-b2e0-4642-98fd-ae03dbd6b644 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:14.829236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:14.829902 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:14.830375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:14.830649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] PATH_INFO: `/v3/users/a4828b30dda248758b4161eafe8668c5` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:14.831802 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4828b30dda248758b4161eafe8668c5)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:14.833747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:14.880192 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4828b30dda248758b4161eafe8668c5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:14.888327 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4828b30dda248758b4161eafe8668c5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:14.889060 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4828b30dda248758b4161eafe8668c5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:14.889777 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4828b30dda248758b4161eafe8668c5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:14.896889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-29cec2ba-78cc-47c4-8a01-597051e027a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4828b30dda248758b4161eafe8668c5'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:14.914269 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1381/2786] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:14 2022] DELETE /identity/v3/users/a4828b30dda248758b4161eafe8668c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:15.085331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.095485 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c2c6e-cc25-4f16-b742-9f70e9204c68 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.134426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:15.157825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.158451 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.158726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.158993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] PATH_INFO: `/v3/projects/be59f6fed9f645d5bd7478535a2fcb02` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.160093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=be59f6fed9f645d5bd7478535a2fcb02)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.162003 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.208421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be59f6fed9f645d5bd7478535a2fcb02', 'request_id': 'req-e5ed1e5a-be95-49ab-a812-f707fa922f5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:15.214542 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be59f6fed9f645d5bd7478535a2fcb02', 'request_id': 'req-e5ed1e5a-be95-49ab-a812-f707fa922f5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:15.221587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be59f6fed9f645d5bd7478535a2fcb02', 'request_id': 'req-e5ed1e5a-be95-49ab-a812-f707fa922f5a'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:15.232215 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] The token cache is being invalidate because project be59f6fed9f645d5bd7478535a2fcb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:15.232495 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:15.233150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e5ed1e5a-be95-49ab-a812-f707fa922f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:15.234632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1406/2787] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:15 2022] DELETE /identity/v3/projects/be59f6fed9f645d5bd7478535a2fcb02 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:15.239498 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.239786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.240065 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.260237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:15.261430 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:15.271655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:15.327169 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1382/2788] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:15.334975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.335255 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.335525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.355835 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:15.357042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:15.364774 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1407/2789] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:15.368921 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.378522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d85666-c315-4eb1-b1c0-d9c0e17e4428 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.383078 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.383669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.383950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.384222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.384866 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:09:15.385232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.386893 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.398522 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1383/2790] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:09:15.403084 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.412112 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72f0b3-dc43-4662-8435-ff08e5866b18 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.417296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.417913 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.418189 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.418458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.419121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.421072 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.456426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1408/2791] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:15 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:15.460595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.470765 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2a748-4773-45f7-b853-845a428dfbf5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.525106 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.526218 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.526497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.526766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.527401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.529315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.550274 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1384/2792] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:15 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:15.554599 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.564972 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06907e-fdf6-4b85-8d85-628d3f577b44 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.569585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.570208 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.570480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.570744 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.571376 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.573209 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.582840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1409/2793] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:15.586725 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.597232 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94c5a3-226b-40f1-8de8-ac319d335662 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.602157 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.602754 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.603032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.603304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.604510 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.606147 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382199a7-23b1-4734-8a85-b02de60fe8df admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.616576 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1385/2794] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:15.621248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.631254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640fe2-ab0f-4b0d-8fbd-b96206aceb7b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.635806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.636442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.636721 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.636991 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] PATH_INFO: `/v3/projects/577429357b754baeb1e053bd5c561bdd/users/1c3767356f594c49b33bf1c79e3b6b63/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.652020 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=577429357b754baeb1e053bd5c561bdd, user_id=1c3767356f594c49b33bf1c79e3b6b63, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.654707 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.669047 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1410/2795] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:15 2022] PUT /identity/v3/projects/577429357b754baeb1e053bd5c561bdd/users/1c3767356f594c49b33bf1c79e3b6b63/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:15.679319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.679601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.679869 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.709534 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] MFA Rules not processed for user `1c3767356f594c49b33bf1c79e3b6b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:15.710846 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:15.773421 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1386/2796] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:15.784993 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:15.786709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:15.838509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:15.860666 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47d1e5-57a0-4bb5-acbf-b16be17ff6d4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:15.866257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:15.866857 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:15.867136 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:15.867408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:15.874990 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:15.876563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c38aa-b524-4883-8d64-d408d22371f1 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:15.885955 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1411/2797] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:18.553011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:18.560627 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:18.614624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2b23e-eb84-45f8-b974-680bcbf9ebce None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:18.619222 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:18.619845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:18.620125 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:18.620397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] PATH_INFO: `/v3/users/1c3767356f594c49b33bf1c79e3b6b63` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:18.621505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3767356f594c49b33bf1c79e3b6b63)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:18.623463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:18.672865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3767356f594c49b33bf1c79e3b6b63'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:18.679598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3767356f594c49b33bf1c79e3b6b63'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:18.680317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3767356f594c49b33bf1c79e3b6b63'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:18.680947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3767356f594c49b33bf1c79e3b6b63'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:18.687990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3767356f594c49b33bf1c79e3b6b63'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:18.706616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1387/2798] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:18 2022] DELETE /identity/v3/users/1c3767356f594c49b33bf1c79e3b6b63 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:18.711075 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:18.711549 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:18.711678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:18.730826 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:18.732045 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:18.790677 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1412/2799] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:18.804115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:18.805595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:18.863462 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70c0bb-ac05-491c-a427-def55e7cfeb9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:18.867140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:18.867740 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:18.868018 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:18.868289 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:18.877481 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:18.879036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfc2b8-446a-43b0-9e5e-17bc860bfdf6 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:18.889150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1388/2800] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:19.155893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.157411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:19.171710 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa866fdb-5d45-4c88-82d9-696316d3ea6d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.177902 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.178723 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.179179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.179576 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] PATH_INFO: `/v3/projects/577429357b754baeb1e053bd5c561bdd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.180817 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=577429357b754baeb1e053bd5c561bdd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.183268 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.230356 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '577429357b754baeb1e053bd5c561bdd', 'request_id': 'req-1a54ce02-c1fa-4553-ab08-7e9805f71e92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:19.236493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '577429357b754baeb1e053bd5c561bdd', 'request_id': 'req-1a54ce02-c1fa-4553-ab08-7e9805f71e92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:19.243271 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '577429357b754baeb1e053bd5c561bdd', 'request_id': 'req-1a54ce02-c1fa-4553-ab08-7e9805f71e92'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:19.253574 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] The token cache is being invalidate because project 577429357b754baeb1e053bd5c561bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:19.253894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:19.254511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-1a54ce02-c1fa-4553-ab08-7e9805f71e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:19.255997 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1413/2801] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:19 2022] DELETE /identity/v3/projects/577429357b754baeb1e053bd5c561bdd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:19.260971 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.261251 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.261519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.280879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:19.282092 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:19.308027 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:19.342636 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1389/2802] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:19.351533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.351819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.352090 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.372031 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:19.373227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:19.380963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1414/2803] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:19.385011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.394485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae099d9-5ab7-4bcc-8924-a34b59afd7af None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.398936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.399519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.399795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.400063 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.401272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:09:19.401568 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.403179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.414408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1390/2804] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:09:19.419056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.427986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990d040-7705-47ab-8334-8107ae4bec4e None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.432616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.433201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.433479 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.433780 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.434406 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.436294 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.465419 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1415/2805] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:19.470097 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.480083 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25f619-b2a3-4483-81b9-2fa5d9adff89 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.535485 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.536117 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.536397 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.536666 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.537351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.539359 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.559660 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1391/2806] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:19.563935 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.574874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091ab63-801f-452b-9555-19f3e9008366 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.579442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.580033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.580313 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.580583 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.581223 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.582850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.593261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1416/2807] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:19.596930 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.606922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f6e1-0eb5-4983-ac75-ce7505d6d07a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.611521 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.612104 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.612381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.612651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.613288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.614934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce851d-289c-4691-819d-a0281561af21 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.625224 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1392/2808] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:09:19.629248 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.639480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357aad-f178-4c78-befb-701e97a1f37d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.644102 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.644735 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.645019 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.645290 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] PATH_INFO: `/v3/projects/22b5d2dee09e41abbcde3c2e98bb5f04/users/a45901bb6ff54fbd93319e95d2dc11fb/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.660312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b5d2dee09e41abbcde3c2e98bb5f04, user_id=a45901bb6ff54fbd93319e95d2dc11fb, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.662966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.670903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:19.676958 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1417/2809] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:19 2022] PUT /identity/v3/projects/22b5d2dee09e41abbcde3c2e98bb5f04/users/a45901bb6ff54fbd93319e95d2dc11fb/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:19.687616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.687905 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.688179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.718112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] MFA Rules not processed for user `a45901bb6ff54fbd93319e95d2dc11fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:19.719323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:19.782000 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1393/2810] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:19.786703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.850205 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669811a-f310-41d5-bcbd-d6cc50ce742a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.854821 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.855426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.855704 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.855974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.856614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.858544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.886027 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1418/2811] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:19.890401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.901610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791466b6-d0dc-4207-a216-7bb4607cd938 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:19.925578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:19.958761 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:19.959362 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:19.959640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:19.959914 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:19.960540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:19.962527 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:19.984017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1394/2812] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:19 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:19.987524 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:19.998470 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aef19-9256-4204-8266-e6866078f3a3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:20.002978 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:20.003571 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.003848 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.004117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.004750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:20.006373 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:20.017173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1419/2813] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:20.021160 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:20.031066 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7065-dc94-4c4c-9599-253f5b0dbc1c admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:20.035681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:20.036344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.036623 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.036896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] PATH_INFO: `/v3/projects/2cab4128872c4ba6a36acf8562ff2501/users/afe570f298e9428faa3f3b9093354089/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.051248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cab4128872c4ba6a36acf8562ff2501, user_id=afe570f298e9428faa3f3b9093354089, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:20.053963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:20.067741 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1395/2814] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:20 2022] PUT /identity/v3/projects/2cab4128872c4ba6a36acf8562ff2501/users/afe570f298e9428faa3f3b9093354089/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:20.071918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:20.082362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9fa6c-ecca-4cf6-b1cb-25b4c1a9f382 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:20.137825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:20.138371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.138649 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.138918 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.139554 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:20.141140 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f87be-d3fb-4b9a-8234-f476c79f5b0f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:20.150740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1420/2815] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:09:20.154990 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:20.170420 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c052467-e3e4-4830-ae2f-21448d9a0b14 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:20.175069 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:20.176148 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.176493 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.176772 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] PATH_INFO: `/v3/projects/2cab4128872c4ba6a36acf8562ff2501/users/afe570f298e9428faa3f3b9093354089/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.178807 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cab4128872c4ba6a36acf8562ff2501, user_id=afe570f298e9428faa3f3b9093354089, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:20.181491 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:20.194091 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1396/2816] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:20 2022] PUT /identity/v3/projects/2cab4128872c4ba6a36acf8562ff2501/users/afe570f298e9428faa3f3b9093354089/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:20.205266 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.205552 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.205867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.234732 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] MFA Rules not processed for user `afe570f298e9428faa3f3b9093354089`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:20.235947 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:20.293051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:20.304433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1421/2817] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:20.317198 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:20.318928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:20.375581 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03ff3-5bde-4704-b01f-c4c6432021fc admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:20.379163 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:20.379751 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:20.380024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:20.380288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:20.389041 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:20.390459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12a189-5bfd-4e44-b92c-6b144f2fbaef service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:20.399574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1397/2818] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:21.715760 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:21.777964 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933392f9-9319-47a3-9285-12ca363a7895 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:21.782460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:21.783083 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:21.783362 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:21.783634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] PATH_INFO: `/v3/users/a45901bb6ff54fbd93319e95d2dc11fb` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:21.784742 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45901bb6ff54fbd93319e95d2dc11fb)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:21.786700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:21.837705 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45901bb6ff54fbd93319e95d2dc11fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:21.845931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45901bb6ff54fbd93319e95d2dc11fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:21.846653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45901bb6ff54fbd93319e95d2dc11fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:21.847740 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45901bb6ff54fbd93319e95d2dc11fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:21.855493 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45901bb6ff54fbd93319e95d2dc11fb'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:21.872886 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1422/2819] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:21 2022] DELETE /identity/v3/users/a45901bb6ff54fbd93319e95d2dc11fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:21.877292 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:21.877786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:21.877923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:21.897974 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:21.899213 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:21.958045 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1398/2820] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:21.967698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:21.969080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:22.024862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112f23b-4f3c-459d-9ea2-599b24578bbe admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:22.028989 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:22.029580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:22.029895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:22.030167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:22.038931 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:22.040364 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:22.050275 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1423/2821] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:09:22.306843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:22.308301 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:22.320349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877deb2-58dc-4df0-9ad4-1690655ae8f8 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:22.325540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:22.326219 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:22.326497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:22.326771 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] PATH_INFO: `/v3/projects/22b5d2dee09e41abbcde3c2e98bb5f04` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:22.327898 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b5d2dee09e41abbcde3c2e98bb5f04)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:22.329843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:22.347260 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:22.377778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b5d2dee09e41abbcde3c2e98bb5f04', 'request_id': 'req-39625b57-9839-4808-b0df-a8c149ea9cdd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.383948 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b5d2dee09e41abbcde3c2e98bb5f04', 'request_id': 'req-39625b57-9839-4808-b0df-a8c149ea9cdd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.391480 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b5d2dee09e41abbcde3c2e98bb5f04', 'request_id': 'req-39625b57-9839-4808-b0df-a8c149ea9cdd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.402315 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] The token cache is being invalidate because project 22b5d2dee09e41abbcde3c2e98bb5f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:22.402597 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.403217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.404652 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1399/2822] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:22 2022] DELETE /identity/v3/projects/22b5d2dee09e41abbcde3c2e98bb5f04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:22.408784 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:22.419262 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec1056-1d3f-4620-a16f-ee679a7ba053 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:22.430511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:22.476756 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:22.477300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:22.477577 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:22.477887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] PATH_INFO: `/v3/users/afe570f298e9428faa3f3b9093354089` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:22.478986 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe570f298e9428faa3f3b9093354089)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:22.480895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:22.530831 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe570f298e9428faa3f3b9093354089'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.537951 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe570f298e9428faa3f3b9093354089'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.538628 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe570f298e9428faa3f3b9093354089'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.539233 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe570f298e9428faa3f3b9093354089'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.546228 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7db33405-0e7f-45f0-9c87-517c1d26f06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe570f298e9428faa3f3b9093354089'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.563587 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1424/2823] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:22 2022] DELETE /identity/v3/users/afe570f298e9428faa3f3b9093354089 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:22.841820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:22.843305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:22.856606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39625b57-9839-4808-b0df-a8c149ea9cdd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:22.913453 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:22.914131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:22.914381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:22.914650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] PATH_INFO: `/v3/projects/2cab4128872c4ba6a36acf8562ff2501` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:22.915766 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cab4128872c4ba6a36acf8562ff2501)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:22.918317 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:22.963255 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cab4128872c4ba6a36acf8562ff2501', 'request_id': 'req-064984cd-8704-4957-8582-5f22c68a3d51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.970072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cab4128872c4ba6a36acf8562ff2501', 'request_id': 'req-064984cd-8704-4957-8582-5f22c68a3d51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.977194 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cab4128872c4ba6a36acf8562ff2501', 'request_id': 'req-064984cd-8704-4957-8582-5f22c68a3d51'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.989644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] The token cache is being invalidate because project 2cab4128872c4ba6a36acf8562ff2501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:22.989961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.990640 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-064984cd-8704-4957-8582-5f22c68a3d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:22.992159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1400/2824] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:22 2022] DELETE /identity/v3/projects/2cab4128872c4ba6a36acf8562ff2501 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:22.997321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:22.997834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:22.997957 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.018604 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:23.019746 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:23.077914 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1425/2825] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:23.085862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.086146 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.086417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.106513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:23.107694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:23.115353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1401/2826] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:23.119361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.128865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3af8c-50c8-4ceb-9ba4-e8cc2d2bebf7 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.133389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.134008 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.134284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.134557 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.135197 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:09:23.135492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.137064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.148308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1426/2827] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:09:23.152923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.161778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269311a9-f7af-4d63-9996-c3782685d45b None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.166302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.166882 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.167769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.168613 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.169256 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.171182 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.197779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1402/2828] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:23.202280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.212055 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa65926-8bd1-4962-86d4-a0d39b385578 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.230662 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:23.268264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.268929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.269138 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.269410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.270093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.271995 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.292965 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1427/2829] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:23.297203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.307478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f2e2-54dd-4a7f-98e4-82d6aa0c3160 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.312141 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.312735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.313011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.313311 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.314112 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.315700 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.316937 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:23.327819 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1403/2830] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:23.331220 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.341625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3a8ac-8e0c-44e1-8206-3038633b19da admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.346444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.347077 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.347357 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.347625 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.348316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.349921 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f0b64-8df7-4e6b-ae98-659cdd2c1ecf admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.360302 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1428/2831] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:09:23.364254 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.374265 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b84fe-ce61-4810-96d0-fb6d6df854cd admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.378803 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.379439 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.379712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.379976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] PATH_INFO: `/v3/projects/5294cd852697433d926894bf49ec4be8/users/d63c9cf6fafb4bc68aad36b80a5fb6bd/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.394651 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5294cd852697433d926894bf49ec4be8, user_id=d63c9cf6fafb4bc68aad36b80a5fb6bd, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.397267 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.410694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1404/2832] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:23 2022] PUT /identity/v3/projects/5294cd852697433d926894bf49ec4be8/users/d63c9cf6fafb4bc68aad36b80a5fb6bd/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:09:23.420962 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.421247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.421518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.448851 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] MFA Rules not processed for user `d63c9cf6fafb4bc68aad36b80a5fb6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:23.450092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:23.512360 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1429/2833] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:23.516872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.578183 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24eb6-a51e-4c0f-84a3-4a3c2591ad2e admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.582845 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.583447 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.583727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.583998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.584625 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.586578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.612280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1405/2834] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:23.616310 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.627633 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de503a05-0a6d-4307-8b5d-35a2917fbdea None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.681999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.682602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.683370 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.683639 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.684276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.686214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.708249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1430/2835] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:23.712153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.722334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b71863-082b-4fb9-8c45-df00346bf13a admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.726888 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.727479 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.727758 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.728048 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.728668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.730309 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.740612 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1406/2836] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:23.744368 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.754646 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fde7f-2b00-40c3-b6b7-7b30a66f486a admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.759207 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.759841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.760115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.760385 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] PATH_INFO: `/v3/projects/86c6315021e14c34ad242efd8d76e85f/users/f0edd5bfbcb04584a97ccaf2c5210768/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.775326 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c6315021e14c34ad242efd8d76e85f, user_id=f0edd5bfbcb04584a97ccaf2c5210768, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.778013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.780945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:23.792265 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1431/2837] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:23 2022] PUT /identity/v3/projects/86c6315021e14c34ad242efd8d76e85f/users/f0edd5bfbcb04584a97ccaf2c5210768/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:09:23.796299 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.806692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193aa65-a693-4225-a2de-d2f9bdbf49e3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.864118 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.864645 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.864923 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.865221 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.865895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.867497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0041a2-0f1b-42da-bf62-ee32040eb9fb admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.877425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1407/2838] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:23.881651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:23.892853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e30f72-9abb-42af-b02b-f4f0d33e2141 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:23.897461 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:23.898145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.898426 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.898696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] PATH_INFO: `/v3/projects/86c6315021e14c34ad242efd8d76e85f/users/f0edd5bfbcb04584a97ccaf2c5210768/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:23.900637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c6315021e14c34ad242efd8d76e85f, user_id=f0edd5bfbcb04584a97ccaf2c5210768, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:23.903336 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:23.913519 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1432/2839] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:23 2022] PUT /identity/v3/projects/86c6315021e14c34ad242efd8d76e85f/users/f0edd5bfbcb04584a97ccaf2c5210768/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:23.976726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:23.977319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:23.977319 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:24.006142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] MFA Rules not processed for user `f0edd5bfbcb04584a97ccaf2c5210768`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:24.007339 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:24.037115 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:24.076242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1408/2840] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:24.089300 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:24.091094 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:24.147610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70dc2d-5e01-4424-9fee-67785a6469b3 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:24.151173 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:24.151768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:24.152048 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:24.152317 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:24.207585 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:24.209023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1145f92-353b-472e-8e80-f37eac6d73bc service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:24.217956 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1433/2841] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:26.741406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:26.802482 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9db18f-096d-4469-b256-d96f21eb9da3 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:26.807067 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:26.807693 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:26.807976 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:26.808257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] PATH_INFO: `/v3/users/d63c9cf6fafb4bc68aad36b80a5fb6bd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:26.809906 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63c9cf6fafb4bc68aad36b80a5fb6bd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:26.811843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:26.856226 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63c9cf6fafb4bc68aad36b80a5fb6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:26.863328 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63c9cf6fafb4bc68aad36b80a5fb6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:26.863947 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63c9cf6fafb4bc68aad36b80a5fb6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:26.864638 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63c9cf6fafb4bc68aad36b80a5fb6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:26.872263 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63c9cf6fafb4bc68aad36b80a5fb6bd'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:26.890081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1409/2842] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:26 2022] DELETE /identity/v3/users/d63c9cf6fafb4bc68aad36b80a5fb6bd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:26.894280 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:26.894709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:26.894819 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:26.913544 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:26.914767 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:26.973722 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1434/2843] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:26.984079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:26.985495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:27.037556 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:27.044344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729b4fe-8592-43ae-b063-b27433e59e7b admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:27.048941 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:27.049549 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:27.049858 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:27.050134 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:27.059855 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:27.061276 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:27.070596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1410/2844] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:27.351010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:27.352468 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:27.364810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa5b7f-a9b7-4494-bb65-f1b0293975db None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:27.369433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:27.370110 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:27.370421 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:27.370698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] PATH_INFO: `/v3/projects/5294cd852697433d926894bf49ec4be8` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:27.371983 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5294cd852697433d926894bf49ec4be8)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:27.373953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:27.411698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:27.426243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5294cd852697433d926894bf49ec4be8', 'request_id': 'req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.432352 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5294cd852697433d926894bf49ec4be8', 'request_id': 'req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.440256 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5294cd852697433d926894bf49ec4be8', 'request_id': 'req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.451804 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] The token cache is being invalidate because project 5294cd852697433d926894bf49ec4be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:27.452092 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.452743 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.454249 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1435/2845] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:27 2022] DELETE /identity/v3/projects/5294cd852697433d926894bf49ec4be8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:27.458321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:27.468235 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74bfcf-9872-414b-8e94-183baf9b819e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:27.522495 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:27.523110 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:27.523387 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:27.523659 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] PATH_INFO: `/v3/users/f0edd5bfbcb04584a97ccaf2c5210768` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:27.524776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0edd5bfbcb04584a97ccaf2c5210768)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:27.526717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:27.572668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0edd5bfbcb04584a97ccaf2c5210768'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.579919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0edd5bfbcb04584a97ccaf2c5210768'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.580603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0edd5bfbcb04584a97ccaf2c5210768'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.581216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0edd5bfbcb04584a97ccaf2c5210768'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.587991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-39219988-ff0e-4472-aa8c-c9d604edd7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0edd5bfbcb04584a97ccaf2c5210768'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:27.605572 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1411/2846] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:27 2022] DELETE /identity/v3/users/f0edd5bfbcb04584a97ccaf2c5210768 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:27.871049 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:27.872582 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:27.885466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5d05b-e0a4-4b61-bd2a-6bc30516c254 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:27.943627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:27.944279 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:27.944575 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:27.944847 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] PATH_INFO: `/v3/projects/86c6315021e14c34ad242efd8d76e85f` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:27.945985 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c6315021e14c34ad242efd8d76e85f)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:27.948198 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:27.994319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c6315021e14c34ad242efd8d76e85f', 'request_id': 'req-429ace33-2036-46fa-ac04-66ea7a0059f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:28.000887 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c6315021e14c34ad242efd8d76e85f', 'request_id': 'req-429ace33-2036-46fa-ac04-66ea7a0059f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:28.007674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c6315021e14c34ad242efd8d76e85f', 'request_id': 'req-429ace33-2036-46fa-ac04-66ea7a0059f2'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:28.018295 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] The token cache is being invalidate because project 86c6315021e14c34ad242efd8d76e85f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:28.018584 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:28.019201 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-429ace33-2036-46fa-ac04-66ea7a0059f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:28.020643 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1436/2847] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:27 2022] DELETE /identity/v3/projects/86c6315021e14c34ad242efd8d76e85f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:28.025472 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.025979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.026099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.045089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:28.046320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:28.103695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1412/2848] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:28.111126 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.111409 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.111679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.131247 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:28.132425 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:28.140816 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1437/2849] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:28.144744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.154216 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf1ffe-d86a-4644-b628-19d0b5f6220a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.158688 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.159271 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.159548 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.159817 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.160461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:09:28.160757 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.162382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.164653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:28.176279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1413/2850] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:09:28.179632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.189444 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341030b-fe46-41b4-8101-e6521409b80f None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.194149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.194738 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.195015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.195283 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.195917 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.198147 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.224600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1438/2851] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:28.228578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.238300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab953e4-3fe3-414b-969b-51f22bd55723 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.293424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.294056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.294340 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.294615 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.295249 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.297167 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.320653 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1414/2852] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:28.322443 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.333227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbe85e-d12d-4341-9705-c040db4568ac admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.337840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.338462 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.338739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.339005 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.339641 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.341226 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.350841 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1439/2853] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:28.354429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.364162 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689bf6-909b-4d70-8766-1659d8ec38e3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.368726 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.369293 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.369574 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.369883 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.370519 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.372956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33db007-0239-4470-9f32-45b11e4a0fd6 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.383670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1415/2854] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:28.388124 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.397882 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3290e3-0ee1-4519-8f1e-5cbe1ff9f64b admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.402509 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.403144 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.403424 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.403698 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] PATH_INFO: `/v3/projects/35c3e8e9363c4d349266b0533aff59b9/users/329879cf32dd4e4abc7bf5c9acea3d54/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.412179 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:28.420925 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c3e8e9363c4d349266b0533aff59b9, user_id=329879cf32dd4e4abc7bf5c9acea3d54, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.423856 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.436513 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1440/2855] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:28 2022] PUT /identity/v3/projects/35c3e8e9363c4d349266b0533aff59b9/users/329879cf32dd4e4abc7bf5c9acea3d54/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:09:28.446523 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.447056 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.447325 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.478610 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] MFA Rules not processed for user `329879cf32dd4e4abc7bf5c9acea3d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:28.480471 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:28.545282 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1416/2856] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:28.549874 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.610569 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7892f-9b30-4df7-93f7-7d3a906fb1a6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.615181 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.615779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.616060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.616334 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.616974 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.618981 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.645530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1441/2857] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:28.649600 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.660237 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80848e-6d7d-420e-983e-be3a2732f2fb None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.717997 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.718601 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.718970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.719243 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.719870 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.721936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.733071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:28.745852 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1417/2858] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:28.750431 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.760665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce176d-8210-4e94-8e99-92c759d702d6 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.765607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.766235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.766518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.766793 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.767436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.769033 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.780038 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1442/2859] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 08:09:28.783733 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.793303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825329-4ae0-49ff-855b-7b5ac91729ff admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.798464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.799047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.799320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.799644 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.800278 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.801876 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df39afb-09ae-4d63-95f1-6273116f1580 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.811582 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1418/2860] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:28.815866 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:28.826782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca9013-2bc5-45f0-b7c0-601f33bce16c admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:28.831810 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:28.832518 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.832825 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.833100 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] PATH_INFO: `/v3/projects/08ca0a4341cf44ff9178ba9d4604d191/users/55246f13a6534b5a97dd926797d0b2ab/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.849133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ca0a4341cf44ff9178ba9d4604d191, user_id=55246f13a6534b5a97dd926797d0b2ab, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:28.851870 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:28.865651 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1443/2861] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:28 2022] PUT /identity/v3/projects/08ca0a4341cf44ff9178ba9d4604d191/users/55246f13a6534b5a97dd926797d0b2ab/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:09:28.877166 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:28.877456 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:28.877752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:28.906980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] MFA Rules not processed for user `55246f13a6534b5a97dd926797d0b2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:28.908185 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:28.971543 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1419/2862] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:28.976009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.038987 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430552a-3dfe-44ce-bc7c-ac0a6a320d91 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.043655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.044261 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.044539 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.044815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.045458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.048062 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.074764 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1444/2863] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:28 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:29.078806 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.089356 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e6361-f7d1-46a6-8cd6-579e7dc277aa None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.145968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.146596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.146873 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.147142 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] PATH_INFO: `/v3/users` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.147776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.149694 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.170963 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1420/2864] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:29 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:09:29.175028 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.194408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ad3c9-d325-404b-9f74-ecf8a19641f8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.202227 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.202903 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.203304 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.203811 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.204979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.207143 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.220616 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1445/2865] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:29.224628 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.234704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dc301-26c9-45bb-9052-49115eeaa4ce admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.239323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.239965 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.240242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.240513 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] PATH_INFO: `/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd/users/60cebf88e087424db0f9532d25b059dd/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.254895 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf8e1b6b15e4c8faa7ad9a26719f7dd, user_id=60cebf88e087424db0f9532d25b059dd, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.257539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.270763 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1421/2866] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:29 2022] PUT /identity/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd/users/60cebf88e087424db0f9532d25b059dd/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:29.274815 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.285401 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d185d-f337-4981-894e-2d0e55a30a03 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.293145 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:29.343270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.343868 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.344148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.344420 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.345053 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.346696 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbd299-5218-44b0-8e4f-b4c4952a3d33 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.356107 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1446/2867] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:29.360229 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.370303 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58fef-78e8-4ff6-916f-2e2bbff22a08 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.374862 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.375500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.375776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.376047 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] PATH_INFO: `/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd/users/60cebf88e087424db0f9532d25b059dd/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.377994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf8e1b6b15e4c8faa7ad9a26719f7dd, user_id=60cebf88e087424db0f9532d25b059dd, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.380643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.391057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1422/2868] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:29 2022] PUT /identity/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd/users/60cebf88e087424db0f9532d25b059dd/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:29.400853 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.401164 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.401445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.429199 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] MFA Rules not processed for user `60cebf88e087424db0f9532d25b059dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:29.430427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:29.502689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1447/2869] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:29.516304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:29.518188 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:29.546679 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:29.583886 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382d87-589b-48ac-abd3-824e8b376720 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:29.587478 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:29.588076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:29.588349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:29.588616 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:29.643239 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:29.644665 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:29.653346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1423/2870] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:31.182218 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:31.183602 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:31.196042 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f0783-ad1e-43b0-a176-fd023f155ba5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:31.200562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:31.201157 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:31.201433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:31.201703 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:31.255773 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:31.257129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:31.266535 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1448/2871] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:31.791727 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:31.801068 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce910-3b0c-4a6d-b239-344dfc9ac6a1 service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:31.804722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:31.805445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:31.805756 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:31.806051 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:31.815748 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:31.817179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1fb3-0eb8-40e1-a5c6-3a582cb86e5a service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:31.827214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1424/2872] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:35.682769 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:35.716087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:35.745319 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165af702-7ee9-47cc-8fcf-7841b62ca97f service cinder] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:35.749879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:35.750512 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:35.750779 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:35.751051 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] PATH_INFO: `/v3/users/329879cf32dd4e4abc7bf5c9acea3d54` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:35.752180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329879cf32dd4e4abc7bf5c9acea3d54)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:35.754192 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:35.799477 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329879cf32dd4e4abc7bf5c9acea3d54'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:35.806335 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329879cf32dd4e4abc7bf5c9acea3d54'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:35.806954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329879cf32dd4e4abc7bf5c9acea3d54'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:35.807563 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329879cf32dd4e4abc7bf5c9acea3d54'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:35.814427 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329879cf32dd4e4abc7bf5c9acea3d54'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:35.831785 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1449/2873] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:35 2022] DELETE /identity/v3/users/329879cf32dd4e4abc7bf5c9acea3d54 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:35.836214 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:35.836715 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:35.836823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:35.856477 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:35.857677 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:35.916773 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1425/2874] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:35.926516 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:35.927922 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:35.984959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a56af-7119-438c-ae69-e2805f2d5de0 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:35.988614 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:35.989246 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:35.989526 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:35.989828 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:35.998982 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:36.000411 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:36.009798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1450/2875] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:36.280054 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:36.281537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:36.297505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867b386-6a80-4a70-b054-cf2c5be0160a None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:36.309646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:36.310150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:36.310429 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:36.310698 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] PATH_INFO: `/v3/projects/35c3e8e9363c4d349266b0533aff59b9` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:36.311912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c3e8e9363c4d349266b0533aff59b9)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:36.313975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:36.359752 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c3e8e9363c4d349266b0533aff59b9', 'request_id': 'req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.365912 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c3e8e9363c4d349266b0533aff59b9', 'request_id': 'req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.374275 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c3e8e9363c4d349266b0533aff59b9', 'request_id': 'req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.375661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:36.388643 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] The token cache is being invalidate because project 35c3e8e9363c4d349266b0533aff59b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:36.388979 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.389614 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.391181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1426/2876] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:36 2022] DELETE /identity/v3/projects/35c3e8e9363c4d349266b0533aff59b9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:36.395243 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:36.404637 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fbd7d-bbb6-42f7-8597-d864a6bccc92 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:36.452095 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:36.462484 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:36.463168 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:36.463446 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:36.463716 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] PATH_INFO: `/v3/users/55246f13a6534b5a97dd926797d0b2ab` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:36.464839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55246f13a6534b5a97dd926797d0b2ab)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:36.466806 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:36.511594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55246f13a6534b5a97dd926797d0b2ab'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.519156 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55246f13a6534b5a97dd926797d0b2ab'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.519937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55246f13a6534b5a97dd926797d0b2ab'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.520691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55246f13a6534b5a97dd926797d0b2ab'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.529953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55246f13a6534b5a97dd926797d0b2ab'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.549506 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1451/2877] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:36 2022] DELETE /identity/v3/users/55246f13a6534b5a97dd926797d0b2ab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:36.812270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:36.813857 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:36.827305 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8fbc9-b4be-4198-bd6a-ab3b3915281d admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:36.883996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:36.884695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:36.884899 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:36.885159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] PATH_INFO: `/v3/projects/08ca0a4341cf44ff9178ba9d4604d191` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:36.886308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ca0a4341cf44ff9178ba9d4604d191)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:36.888489 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:36.936171 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ca0a4341cf44ff9178ba9d4604d191', 'request_id': 'req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.942386 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ca0a4341cf44ff9178ba9d4604d191', 'request_id': 'req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.949993 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ca0a4341cf44ff9178ba9d4604d191', 'request_id': 'req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.960722 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] The token cache is being invalidate because project 08ca0a4341cf44ff9178ba9d4604d191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:36.961009 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.961699 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:36.963291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1427/2878] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:36 2022] DELETE /identity/v3/projects/08ca0a4341cf44ff9178ba9d4604d191 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:36.967341 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:36.968741 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:36.981822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc875-bcab-4156-afd3-c9a0a69ce481 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.039388 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.040010 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.040287 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.040559 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] PATH_INFO: `/v3/users/60cebf88e087424db0f9532d25b059dd` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.041671 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cebf88e087424db0f9532d25b059dd)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.044296 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.088959 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cebf88e087424db0f9532d25b059dd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.096650 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cebf88e087424db0f9532d25b059dd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.097429 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cebf88e087424db0f9532d25b059dd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.098103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cebf88e087424db0f9532d25b059dd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.105162 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-e0e36da1-23e7-4b53-b57a-e248ef74c659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cebf88e087424db0f9532d25b059dd'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.113553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:37.124436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1452/2879] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:09:36 2022] DELETE /identity/v3/users/60cebf88e087424db0f9532d25b059dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:37.389515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.390968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:37.405931 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cdad4-d8b0-48b7-8a99-db6237d7d368 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.466894 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.467539 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.467815 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.468131 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] PATH_INFO: `/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.469246 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf8e1b6b15e4c8faa7ad9a26719f7dd)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.471208 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.493322 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:37.518991 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf8e1b6b15e4c8faa7ad9a26719f7dd', 'request_id': 'req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.525199 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf8e1b6b15e4c8faa7ad9a26719f7dd', 'request_id': 'req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.533153 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf8e1b6b15e4c8faa7ad9a26719f7dd', 'request_id': 'req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.543896 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] The token cache is being invalidate because project fbf8e1b6b15e4c8faa7ad9a26719f7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:09:37.544205 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.544928 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-e57ac0d9-ab2c-4b46-a9ec-85e024024a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:09:37.546508 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1428/2880] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:09:37 2022] DELETE /identity/v3/projects/fbf8e1b6b15e4c8faa7ad9a26719f7dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:37.551445 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.551751 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.552023 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.571041 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:37.572480 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:37.630253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1453/2881] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:37.639996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.640280 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.640550 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.660802 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:37.662061 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:37.669843 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1429/2882] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:37.674064 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.683566 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd289a6e-1bfb-406d-8564-01bd8ccc60e5 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.688347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.688977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.689253 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.689523 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.690235 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:09:37.690537 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.692264 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.705442 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1454/2883] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:09:37.709713 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.719704 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b028-a0e4-48a6-96c8-cbe739864e79 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.724277 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.724861 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.725139 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.725405 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.726076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.727967 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.754879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1430/2884] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:09:37 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:37.759006 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.769377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b388c3c-e52a-401e-81dd-4d4d116c50d1 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.823709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.824436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.825346 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.825615 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.826276 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.828167 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.849093 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1455/2885] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:09:37 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:09:37.853320 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.863353 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6556c0c-0e7d-46f5-bd80-e98cd3d008c0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.867879 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.868464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.868738 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.869006 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.869641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.871258 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.881351 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1431/2886] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:37.885241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.895850 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26a30c-8b78-46bf-a9f0-18d1a8bf3abd admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.900507 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.901086 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.901365 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.901634 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.902312 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.904413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd619eef-8be6-454c-81e6-3b53d49e49e3 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.915839 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1456/2887] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:09:37.920326 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:37.930557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f0b86-1f77-4fd4-b87a-4978fedaa6b9 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:37.935735 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:37.936381 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.936661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.936936 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] PATH_INFO: `/v3/projects/03c33711cc664b1685882efa8d15cce4/users/07480e5645e143c199389fce5b375f98/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.952164 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c33711cc664b1685882efa8d15cce4, user_id=07480e5645e143c199389fce5b375f98, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:37.954994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:37.969400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1432/2888] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:09:37 2022] PUT /identity/v3/projects/03c33711cc664b1685882efa8d15cce4/users/07480e5645e143c199389fce5b375f98/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:09:37.974663 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727e98-18e9-4a91-82b4-5f9b71f339f6 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:37.975214 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727e98-18e9-4a91-82b4-5f9b71f339f6 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:37.975316 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44727e98-18e9-4a91-82b4-5f9b71f339f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:37.993591 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-44727e98-18e9-4a91-82b4-5f9b71f339f6 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:37.994798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-44727e98-18e9-4a91-82b4-5f9b71f339f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:38.043759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:38.054448 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1457/2889] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:09:38.064378 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:38.066086 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:38.123143 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b9711-27f3-4a19-a246-29a5e5a0bbd5 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:38.126891 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:38.127492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:38.127769 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:38.128036 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:38.136865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:38.138919 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:38.148181 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1433/2890] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:38.736963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0e8a-95c3-49b7-b624-855de7a1de8b None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:38.737291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0e8a-95c3-49b7-b624-855de7a1de8b None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:38.737530 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b0e8a-95c3-49b7-b624-855de7a1de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:38.765794 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-165b0e8a-95c3-49b7-b624-855de7a1de8b None None] MFA Rules not processed for user `07480e5645e143c199389fce5b375f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:38.767025 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-165b0e8a-95c3-49b7-b624-855de7a1de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:38.834440 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1458/2891] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:38.846159 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:38.847537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:38.907138 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:38.911779 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f5568-e882-4644-a59a-7afc7f76ca58 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:38.916089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8e54e867eee421b8f230c259d3cce68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:38.916922 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:38.917646 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:38.917980 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:38.927017 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:38.928450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:38.937304 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1434/2892] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:40.296059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:40.296572 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:40.296689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:40.315137 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] MFA Rules not processed for user `07480e5645e143c199389fce5b375f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:09:40.316322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:40.323274 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1459/2893] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:09:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:09:40.558557 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:40.559989 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:40.572126 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63401b-0c17-4368-88e2-c254d27744fc service cinder] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:40.575669 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:40.576747 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:40.577024 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:40.577295 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:40.578975 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:40.588133 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:40.589573 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:40.598227 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1435/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:40.911765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:40.913321 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:40.988111 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b0839-8aab-4ac8-a87b-a801fca57b9d None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:40.992966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:40.993573 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:40.993879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:40.994175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:40.995970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:41.007759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:41.009413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:41.022459 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1460/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:42.652986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:42.661872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f55296-3a9e-4867-a095-337b46a838b8 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:42.665505 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:42.666145 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:42.666423 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:42.666695 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:42.668413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:42.678307 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:42.679717 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e32c7-af39-454f-9e92-d7103a3f9d17 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:42.688377 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1436/2896] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:09:47.580879 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:09:47.582378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:47.658692 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0069803-2331-4f39-90aa-20665836ccb0 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:09:47.664337 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:09:47.664954 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:09:47.665236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:09:47.665511 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:09:47.667331 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:09:47.677603 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=100 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:47.680059 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:09:47.682270 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:09:47.692979 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1461/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:09:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:09:52.684156 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=16 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:09:57.688434 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:10:36.858956 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:10:36.859524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:10:36.859524 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:10:36.878823 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] MFA Rules not processed for user `07480e5645e143c199389fce5b375f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:10:36.879996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:10:36.887349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1437/2898] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:10:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:10:46.873336 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:30.102433 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.113586 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342bbb0-46ff-48aa-85e6-d0690d76883e service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.118620 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.119134 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.119412 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.119682 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] PATH_INFO: `/v3/users/07480e5645e143c199389fce5b375f98` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.120802 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] RBAC: Authorizing `identity:delete_user(user_id=07480e5645e143c199389fce5b375f98)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.122736 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.177105 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07480e5645e143c199389fce5b375f98'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.184142 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07480e5645e143c199389fce5b375f98'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.184798 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07480e5645e143c199389fce5b375f98'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.185400 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07480e5645e143c199389fce5b375f98'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.192291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-5824b1ef-efe2-4c5d-aa64-097d0c12651d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07480e5645e143c199389fce5b375f98'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.210074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1462/2899] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:11:30 2022] DELETE /identity/v3/users/07480e5645e143c199389fce5b375f98 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:11:30.377179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.388408 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f9583-b381-4d67-8118-60014ad09060 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.444245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.444801 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.445081 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.445350 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] PATH_INFO: `/v3/projects/03c33711cc664b1685882efa8d15cce4` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.446496 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c33711cc664b1685882efa8d15cce4)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.448692 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.494296 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c33711cc664b1685882efa8d15cce4', 'request_id': 'req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.500445 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c33711cc664b1685882efa8d15cce4', 'request_id': 'req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.509150 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c33711cc664b1685882efa8d15cce4', 'request_id': 'req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.520418 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] The token cache is being invalidate because project 03c33711cc664b1685882efa8d15cce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81452) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:11:30.520674 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.521338 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-c17ff4c2-894e-4dd2-81d0-f64e77a6ff3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:11:30.522820 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1438/2900] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:11:30 2022] DELETE /identity/v3/projects/03c33711cc664b1685882efa8d15cce4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:11:30.527709 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.527994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.528267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.547713 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:30.548900 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:30.607439 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1463/2901] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 08:11:30.615098 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.615380 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.615681 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.635272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:30.636448 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:30.644786 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1439/2902] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:11:30.648765 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.658379 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd344e-632d-4abb-9d2d-aa2cf0e0d466 None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.662862 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.663452 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.663729 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.663998 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] PATH_INFO: `/v3/domains` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.664640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81451) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 08:11:30.664942 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.666579 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.677999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1464/2903] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 08:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 08:11:30.682257 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.691875 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de7ba-268d-4361-8e3a-59955ac44fcd None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.696540 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.697334 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.697622 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.697927 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.698558 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.700464 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.728961 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1440/2904] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:11:30 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:11:30.733069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.742976 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a682c8-ec8b-4fb0-82e5-4d2fbdd3b053 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.798939 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.799548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.799876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.800148 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.800786 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.802739 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.812733 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:30.828679 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1465/2905] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:11:30 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:11:30.833642 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.843833 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47684b8d-880d-4e58-9b4f-c47a897a3da1 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.848463 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.849057 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.849332 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.849598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.850308 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.851884 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.861648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1441/2906] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:11:30.865759 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.875436 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184ed8b-fa1b-499a-936d-caf85f6ae8b8 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.880545 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.881131 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.881407 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.881678 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] PATH_INFO: `/v3/roles` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.882347 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.883928 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a6987-5389-4ae2-a326-956556c1f10e admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.894640 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1466/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:11:30.898607 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:30.908432 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8faa6f3-314d-498e-9d74-eb5b0344b5c3 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:30.912986 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:30.913654 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.913959 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.914230 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] PATH_INFO: `/v3/projects/9b574ef95c3547b39cabd8623717e508/users/10ddd504a9654f16b8c813c91d573306/roles/6e7afd6453c542aaa20fbb52594a1522` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.928841 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b574ef95c3547b39cabd8623717e508, user_id=10ddd504a9654f16b8c813c91d573306, role_id=6e7afd6453c542aaa20fbb52594a1522)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:30.931492 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:30.944648 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1442/2908] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:11:30 2022] PUT /identity/v3/projects/9b574ef95c3547b39cabd8623717e508/users/10ddd504a9654f16b8c813c91d573306/roles/6e7afd6453c542aaa20fbb52594a1522 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:11:30.949937 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1a20-4973-4780-b2a0-e44aa8afe46d None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:30.950289 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1a20-4973-4780-b2a0-e44aa8afe46d None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:30.950570 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1a20-4973-4780-b2a0-e44aa8afe46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:30.968975 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-e9db1a20-4973-4780-b2a0-e44aa8afe46d None None] MFA Rules not processed for user `d247b00da71b4a5ebefbb06a796143be`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:30.970190 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e9db1a20-4973-4780-b2a0-e44aa8afe46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:31.026299 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1467/2909] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:11:31.036120 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:31.037812 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:31.068391 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:31.097849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e88060-51d6-41d8-be19-7ae8bacadb7f admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:31.101497 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:31.102140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:31.102419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:31.102689 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:31.111569 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:31.113490 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:31.123375 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1443/2910] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:11:35.098580 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.099466 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.099877 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.132103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] MFA Rules not processed for user `10ddd504a9654f16b8c813c91d573306`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:35.133689 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:35.203861 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1468/2911] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:11:35.208565 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.218401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb47e28-bc01-4f31-9ee0-691f672f0c4e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.224190 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.224943 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.225288 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.225661 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] PATH_INFO: `/v3/projects` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.226425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.228409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.257780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1444/2912] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 08:11:35 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 08:11:35.262150 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.273909 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0c962-8d38-4faf-aff6-7501424fd5ed None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.334548 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.335250 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.335748 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.336078 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] PATH_INFO: `/v3/users` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.336790 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.339202 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.356060 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:35.364013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1469/2913] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 08:11:35 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 08:11:35.367639 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.378079 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0207c-9947-4e3e-8001-1b4457d86840 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.382719 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.383323 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.383598 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.383865 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.384507 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.386140 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.398039 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1445/2914] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:11:35.402230 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.413269 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9de69-23a1-4c7d-bed5-f7f35715c17d admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.418081 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.418754 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.419034 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.419308 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] PATH_INFO: `/v3/projects/f0204c72b99141b38bde74c3482a67e0/users/e70d576d406649b7a331a1139e548f4c/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.435687 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0204c72b99141b38bde74c3482a67e0, user_id=e70d576d406649b7a331a1139e548f4c, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.438608 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.452492 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1470/2915] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:11:35 2022] PUT /identity/v3/projects/f0204c72b99141b38bde74c3482a67e0/users/e70d576d406649b7a331a1139e548f4c/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:11:35.458406 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.470349 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34a12b-a42c-4ff0-92c9-cfeb61f7aef0 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.529089 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.529702 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.530010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.530279 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.530944 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.532579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa9f1f-7662-45a1-bcbe-73f79208eb9b admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.543371 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1446/2916] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 08:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 08:11:35.547659 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:35.559598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ade13d-f30d-4901-bd5d-b1635117d25f admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:35.564456 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:35.565117 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:35.565428 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:35.565867 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] PATH_INFO: `/v3/projects/f0204c72b99141b38bde74c3482a67e0/users/e70d576d406649b7a331a1139e548f4c/roles/7b12b864f5fc46169140cae8eff50f57` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:35.567865 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0204c72b99141b38bde74c3482a67e0, user_id=e70d576d406649b7a331a1139e548f4c, role_id=7b12b864f5fc46169140cae8eff50f57)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:35.570647 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:35.582284 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1471/2917] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 08:11:35 2022] PUT /identity/v3/projects/f0204c72b99141b38bde74c3482a67e0/users/e70d576d406649b7a331a1139e548f4c/roles/7b12b864f5fc46169140cae8eff50f57 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:11:40.021187 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85c15-4376-4782-b947-f8e6ff54ca28 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:40.021671 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85c15-4376-4782-b947-f8e6ff54ca28 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:40.021805 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c85c15-4376-4782-b947-f8e6ff54ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:40.071209 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-f7c85c15-4376-4782-b947-f8e6ff54ca28 None None] MFA Rules not processed for user `e70d576d406649b7a331a1139e548f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:40.072816 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-f7c85c15-4376-4782-b947-f8e6ff54ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:40.157252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1447/2918] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:11:40.180525 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:11:40.182273 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:40.271139 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5fcf1-4f39-4eb3-a75b-16a8c138c438 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:11:40.276895 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:11:40.277655 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:40.277977 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:40.278259 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:40.338069 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:11:40.339665 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:11:40.350174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1472/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:11:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:11:50.153962 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:50.345691 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:11:55.614413 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:11:55.614934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:11:55.614934 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:11:55.634425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] MFA Rules not processed for user `10ddd504a9654f16b8c813c91d573306`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:11:55.635611 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:11:55.642401 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1448/2920] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:11:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:12:05.629203 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=3 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:12:10.745089 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:12:10.799864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ddeb-751b-42b4-9efc-b5a36f20007e service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:12:10.803999 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:12:10.804600 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:12:10.804876 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:12:10.805149 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:12:10.814121 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:12:10.815546 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:12:10.825700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1473/2921] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:12:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:12:12.144461 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:12:12.146424 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:12:12.216071 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2241a82-5525-4721-b94a-70b0020bc507 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:12:12.221032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:12:12.221641 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:12:12.221973 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:12:12.222245 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:12:12.224551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:12:12.236128 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:12:12.237707 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:12:12.247839 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1449/2922] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:12:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:12:12.377371 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:12:12.378824 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:12:12.401080 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feacbb0-e6ed-4113-b3e7-1fd3626369e9 service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:12:12.410015 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:12:12.410656 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:12:12.410941 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:12:12.411257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:12:12.413129 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:12:12.426726 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:12:12.428135 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:12:12.439632 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1474/2923] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:12:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:12:22.243403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=32 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:12:22.434533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=32 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:14:08.130978 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:14:08.140872 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650bb8e-3a92-499f-a570-961934f2c7b0 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:14:08.145449 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:14:08.146076 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:14:08.146354 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:14:08.146624 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:14:08.148344 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:14:08.162593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:14:08.164025 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2468753-9e09-48c4-994d-5404f9349a39 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:14:08.175049 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1450/2924] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:14:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:14:09.352403 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:14:09.353768 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:14:09.365702 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454b62-b7f6-41b9-a076-b5e49db77009 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:14:09.369267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:14:09.369894 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:14:09.370175 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:14:09.370447 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:14:09.372079 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:14:09.385151 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:14:09.386597 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:14:09.397963 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1475/2925] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:14:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:14:11.696682 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] REQUEST_METHOD: `POST` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:14:11.697127 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:14:11.697385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:14:11.718712 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.auth.core [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] MFA Rules not processed for user `10ddd504a9654f16b8c813c91d573306`. Rule list: `[]` (Enabled: `True`). {{(pid=81452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:14:11.719929 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:14:11.729551 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1451/2926] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:14:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:14:12.043653 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:14:12.052855 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d01b7b-6f4c-4815-9b9c-85e1478dee3a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:14:12.056361 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:14:12.056953 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:14:12.057232 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:14:12.057505 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:14:12.059257 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:14:12.068822 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:14:12.070291 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43aca4d-e7aa-4e77-8fcd-01ac46cc8f12 service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:14:12.078893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1476/2927] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:14:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:14:21.713720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=9 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:14:22.075114 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=12 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:15:16.887008 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:15:16.896970 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0ac44-0d12-4a5c-bcd1-97e4e2029fa9 None None] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:15:16.900500 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:15:16.901099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:15:16.901400 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:15:16.901760 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:15:16.903450 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:15:16.913300 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:15:16.914750 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:15:16.922998 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1452/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:15:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:15:26.919217 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=6 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:16:15.479396 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] REQUEST_METHOD: `POST` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:16:15.479929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:16:15.479929 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:16:15.498700 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.auth.core [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] MFA Rules not processed for user `10ddd504a9654f16b8c813c91d573306`. Rule list: `[]` (Enabled: `True`). {{(pid=81451) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 08:16:15.499904 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:15.506893 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1477/2929] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 08:16:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 08:16:15.823459 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:16:15.832075 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e42b37-82ee-4c08-8667-819ceff0196e service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:16:15.836417 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:16:15.837010 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:16:15.837291 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:16:15.837561 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:16:15.839290 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:15.848670 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:16:15.850132 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:16:15.858135 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1453/2930] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:16:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:16:16.290610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:16:16.292174 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:16.307970 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902709a5-dd50-4d8a-b8b4-40ef6f0673ca None None] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:16:16.312864 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:16:16.313594 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:16:16.313910 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:16:16.314180 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:16:16.315873 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:16.328503 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:16:16.330115 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:16:16.340408 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1478/2931] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:16:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:16:18.766443 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:16:18.781667 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f800-1ae6-4b48-9a76-aafa17400c59 service neutron] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:16:18.787286 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:16:18.788005 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:16:18.788248 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:16:18.788469 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:16:18.790242 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:18.802744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:16:18.804385 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:16:18.815321 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1454/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:16:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:16:26.337241 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=9 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:16:28.109050 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:16:28.126074 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd497294-cd59-46f2-9a53-5df4986e038b service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:16:28.131966 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:16:28.132699 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:16:28.132840 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:16:28.133152 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:16:28.135619 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:16:28.148487 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:16:28.150122 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:16:28.171414 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1479/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:16:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:16:28.811346 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=12 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:16:38.156343 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=6 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:17:13.131732 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:17:13.190668 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6746-723b-44ac-9a79-3f9f80d14a84 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:17:13.195720 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:17:13.196302 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:17:13.196579 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:17:13.196849 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:17:13.205650 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:17:13.207252 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:17:13.217795 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1455/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:17:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:17:23.213563 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=27 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:18:00.312322 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:00.348481 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549725d4-76bf-41b0-aba1-95a07b71d380 service nova] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:00.352195 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:00.352795 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:00.353070 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:00.353344 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:00.361533 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:00.362984 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:00.372087 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1480/2935] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:18:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:18:10.309595 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:10.341220 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfb9c3-ec57-42a7-ad54-f2281f392c3a service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:10.346736 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '742f83bd38b542d193a911ef623abd45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:10.347360 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:10.347603 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:10.347939 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:10.357215 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:10.358825 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:10.368014 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=18 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:18:10.369409 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1456/2936] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:18:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:18:13.768627 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:13.777813 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7948c-3044-4d82-9299-90b48a9ec12a service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:13.781460 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0abf2c4f1a45b88cf4f54237eb0def', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:13.782103 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:13.782378 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:13.782648 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:13.852056 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:13.853502 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:13.873097 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1481/2937] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 08:18:16.892701 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:16.918013 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27adf62-16da-48e0-9906-1f52431418d2 service nova] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:16.924503 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:16.925516 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:16.925984 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:16.926419 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:16.928968 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:18:16.985119 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:16.988072 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:17.000034 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1457/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:18:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:18:20.029553 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:20.041562 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2232c8-1af6-491e-8294-d6e11421d6af service neutron] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:20.047834 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:20.048610 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] REQUEST_METHOD: `GET` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:20.048996 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:20.049510 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:20.051674 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:18:20.065245 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:20.067236 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:20.090099 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:20.093496 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1482/2939] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:18:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:18:20.101272 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113bb1b-2b47-460f-9e4d-24d65eaf0d14 service placement] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:20.106828 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:20.107515 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:20.107874 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:20.108210 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] PATH_INFO: `/v3/users/10ddd504a9654f16b8c813c91d573306` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:20.109403 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ddd504a9654f16b8c813c91d573306)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:20.111415 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:20.158383 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ddd504a9654f16b8c813c91d573306'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.165778 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ddd504a9654f16b8c813c91d573306'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.166425 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ddd504a9654f16b8c813c91d573306'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.167606 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ddd504a9654f16b8c813c91d573306'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.174578 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ddd504a9654f16b8c813c91d573306'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.192596 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1458/2940] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:18:20 2022] DELETE /identity/v3/users/10ddd504a9654f16b8c813c91d573306 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:18:20.362778 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:20.373458 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a2c4-f2f5-402a-b1fa-cc36452ec4e2 service placement] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:20.433052 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:20.433668 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:20.434009 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:20.434282 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] PATH_INFO: `/v3/projects/9b574ef95c3547b39cabd8623717e508` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:20.435380 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b574ef95c3547b39cabd8623717e508)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:20.437254 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:20.482133 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b574ef95c3547b39cabd8623717e508', 'request_id': 'req-fa8b76c6-7287-41b8-b7ed-06a305313fb4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.488211 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b574ef95c3547b39cabd8623717e508', 'request_id': 'req-fa8b76c6-7287-41b8-b7ed-06a305313fb4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.495410 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b574ef95c3547b39cabd8623717e508', 'request_id': 'req-fa8b76c6-7287-41b8-b7ed-06a305313fb4'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.499166 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:18:20.519598 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] The token cache is being invalidate because project 9b574ef95c3547b39cabd8623717e508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:18:20.519888 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.520514 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.522013 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1483/2941] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:18:20 2022] DELETE /identity/v3/projects/9b574ef95c3547b39cabd8623717e508 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 08:18:20.526690 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:20.528179 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:18:20.540933 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a51c82-7a13-4c12-a5e8-c67546084da2 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:20.595382 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:20.595996 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:20.596270 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:20.596537 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] PATH_INFO: `/v3/users/e70d576d406649b7a331a1139e548f4c` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:20.597649 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70d576d406649b7a331a1139e548f4c)` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:20.599584 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:20.612137 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:18:20.644780 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70d576d406649b7a331a1139e548f4c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.652655 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70d576d406649b7a331a1139e548f4c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.653593 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70d576d406649b7a331a1139e548f4c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.654281 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70d576d406649b7a331a1139e548f4c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.661465 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.notifications [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70d576d406649b7a331a1139e548f4c'} {{(pid=81452) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.678444 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1459/2942] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 08:18:20 2022] DELETE /identity/v3/users/e70d576d406649b7a331a1139e548f4c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:18:20.851101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Authenticating user token {{(pid=81451) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:18:20.852607 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.fernet_utils [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81451) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:18:20.868120 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b76c6-7287-41b8-b7ed-06a305313fb4 admin admin] Validating token access rules against request {{(pid=81451) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:18:20.927750 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247b00da71b4a5ebefbb06a796143be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab10ae977dc434f9d22b1fb83ca19e8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81451) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:18:20.928389 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:18:20.928672 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:18:20.928945 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] PATH_INFO: `/v3/projects/f0204c72b99141b38bde74c3482a67e0` {{(pid=81451) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:18:20.930101 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0204c72b99141b38bde74c3482a67e0)` {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:18:20.932377 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] RBAC: Authorization granted {{(pid=81451) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:18:20.982267 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0204c72b99141b38bde74c3482a67e0', 'request_id': 'req-de7d401a-2af1-4e70-a61f-9e88a76d8aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.988711 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0204c72b99141b38bde74c3482a67e0', 'request_id': 'req-de7d401a-2af1-4e70-a61f-9e88a76d8aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:20.998413 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0204c72b99141b38bde74c3482a67e0', 'request_id': 'req-de7d401a-2af1-4e70-a61f-9e88a76d8aed'} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:21.009994 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] The token cache is being invalidate because project f0204c72b99141b38bde74c3482a67e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81451) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 08:18:21.011113 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:21.011782 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG keystone.notifications [None req-de7d401a-2af1-4e70-a61f-9e88a76d8aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81451) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 08:18:21.013277 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: [pid: 81451|app: 0|req: 1484/2943] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 08:18:20 2022] DELETE /identity/v3/projects/f0204c72b99141b38bde74c3482a67e0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 08:18:30.674994 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:18:31.006908 dv-cinder-857075-3-iscsi devstack@keystone.service[81451]: DEBUG dbcounter [-] [81451] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=81451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 08:22:15.133032 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Authenticating user token {{(pid=81452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 08:22:15.134950 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:22:15.204011 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3125bd-7d0a-4281-8de9-78e244b298d7 admin admin] Validating token access rules against request {{(pid=81452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 08:22:15.262632 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6fd2b2ee55f419195b935ba13743dc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a0991faba72483b98d9ff2d19d1e952', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 08:22:15.263236 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] REQUEST_METHOD: `GET` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 08:22:15.263511 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] SCRIPT_NAME: `/identity` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 08:22:15.263776 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 08:22:15.265476 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.fernet_utils [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 08:22:15.277847 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 08:22:15.279889 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040eebf-b361-4f5e-922c-873ffe2850de service placement] RBAC: Authorization granted {{(pid=81452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 08:22:15.290023 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: [pid: 81452|app: 0|req: 1460/2944] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 08:22:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 08:22:25.285744 dv-cinder-857075-3-iscsi devstack@keystone.service[81452]: DEBUG dbcounter [-] [81452] Writing DB stats keystone:SELECT=58 {{(pid=81452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}